Django + WordPress.com REST API =
PROFIT
Jeff Sternberg, VP TechObserver [email protected]://www.linkedin.com/in/jeffsternberg@sternb0t
About me
- Lead engineer @Observer < 1 year- NYC fintech startups ~ 2 years- S&P Capital IQ ~ 10 years- Once and future data scientist- Python brings me joy, SQL makes me happy- First code-for-pay: AppleScript
About Observer- Founded in 1987- We publish an actual
printed newspaper, the weekly New York Observer
- Observer.com- PolitickerNJ.com- CommercialObserver.com
Commercial Observer
- Commercial real estate- Leasing, sales, financing, construction,
infrastructure, industry players, features- Weekly print edition
- Strong since inception 5 years ago- Digital edition
- Needed some love
Before..
After!
Setting the CO Scene
- Small in-house editorial team- Handful of contributors (industry experts)- WordPress codebase, launched years ago* - Not modified much since launch- Hosted on WordPress.com VIP
*I don’t actually know when it was launched
WordWhat?
Though it powers 25% of the internet, I hadn’t ever built anything with WordPress.
I didn’t even know PHP.
WordPress is actually pretty good
- Great WYSIWYG post editor- Easy for editors and writers to learn- Decent media library- Handles most publishing use cases- Lots of useful plugins- Large developer community
Except...
- Rigid data model: everything’s a Post- Not MVC- Plugin spaghetti- Most WP sites are hobbyist/small blogs;
enterprise WP community is fairly small- Locked-down VIP hosting optimized for
high traffic sites like observer.com
Let’s build this in Django.
Ok, maybe just the front end.
Django / WordPress integration
Option 1: Django connects to WordPress db
WordPress content db(mysql)
WordPress web app (php) Django web app (python)
HTTP/HTML HTTP/HTML
TCP (data)
Django / WordPress integration
Option 1: Django connects to WordPress db- sunlightlabs/django-wordpress- agiliq/django-wordpress- Or, roll your own models
- WordPress only has ~12 tables
Django / WordPress integration
Option 2: Coupled REST API
WordPress content dbmysql
WordPress web app (php) Django web app (python)
HTTP/HTML HTTP/HTML
HTTP/REST
Django / WordPress integration
Option 2: Coupled REST API- WordPress.com REST API
- https://developer.wordpress.com/docs/api/- WordPress.org: use WP REST API
- http://v2.wp-api.org/
Django / WordPress integration
Option 3: Decoupled REST API
WordPress content dbmysql
WordPress web app (php) Django web app (python)
HTTP/HTML HTTP/HTML
HTTP/REST
Django db
Django / WordPress integration
Option 3: Decoupled REST API- Same WP REST APIs- Django has its own copy of the content db- Sync content with cron + webhooks
Shared db vs. REST API
Shared db- Faster data access- Django should be
in the same network as WP
- Read/write?
REST API- Django can use
separate hosting / network
- Higher latency- Data serialization /
deserialization
Coupled vs. Decoupled REST API
Coupled- Only 1 copy of the
content db- Site uptime
depends on both Django and WP
Decoupled- Can easily alter /
extend db schema- Data sync scripts
can be tricky- Hosting flexibility
For WordPress.org sites
For WordPress.com sites
WordPress.com REST API
- Sane data model- /v1.1/sites/$site_id/posts- /v1.1/sites/$site_id/tags- etc.
- OAuth2 for private data (e.g. draft posts)- Handy dev console- Good documentation- Some gotchas
Gotchas: WordPress.com REST APIWhitelist custom post types in theme code:/** Allow additional post types in wp.com REST API */function obs_rest_api_post_types( $allowed_post_types ) { $allowed_post_types[] = 'guest-author'; $allowed_post_types[] = 'attachment'; $allowed_post_types[] = 'sponsored-post'; return $allowed_post_types;}add_filter( 'rest_api_allowed_post_types', 'obs_rest_api_post_types' );
Gotchas: WordPress.com REST API
- Custom taxonomies are not supported: the only post terms you can fetch are tags and categories
- So we can use post meta as a workaround...
/* Save coauthors slugs in meta */function obs_save_coauthor_tax_as_meta( $post_id ) { $authors = get_coauthors( $post_id ); $author_str = json_encode( array_map(
function( $a ) { return 'cap-' . $a; }, wp_list_pluck( $authors, 'user_nicename' ) ) );
if ( $author_str ) { update_post_meta( $post_id, 'nyo-cap-slug', $author_str ); } else { delete_post_meta( $post_id, 'nyo-cap-slug' ); }}
add_action( 'save_post', 'obs_save_coauthor_tax_as_meta' );
Decoupled data syncing
- Django command (cron)
- Webhook
$ python manage.py load_wp_api
$ curl -X POST --data "ID=281878" http://co/api/story
import requestsfrom django.conf import settings
def load_posts(site_id, post_type):
api_url = "https://public-api.wordpress.com/rest/v1.1/" \ "sites/{}/posts".format(site_id)
headers = {"Authorization": "Bearer {}".format(settings.WP_API_TOKEN)}
params = {"number": 100, "type": post_type} page = 1
Django command (cron)
def load_posts(site_id, post_type): # ...
# set modified_after to "continue where we left off" latest = Post.objects.filter(post_type=post_type)\ .order_by("-modified")\ .first() if latest: params["modified_after"] = latest.modified.isoformat()
# get first page response = requests.get(api_url, params=params, headers=headers)
Django command (cron) continued
def load_posts(site_id, post_type): # ...
while response.ok and response.text: api_json = response.json() api_posts = api_json.get("posts") for api_post in api_posts: load_wp_post(site_id, api_post) # helper function next_page_handle = api_json.get("meta", {})\ .get("next_page") if next_page_handle: params["page_handle"] = next_page_handle else: break # no more pages left response = requests.get(api_url, params=params, headers=headers)
from rest_framework.views import APIViewfrom rest_framework.response import Response
class LoadAPIStoryView(APIView): def post(self, request): try: wp_id = int(request.POST["ID"]) except: raise Http404("Post does not exist") load_story.after_response(request, wp_id) return Response({"status": "loading wp_id: {}".format(wp_id)})
Webhook
import after_responsefrom django.conf import settingsfrom wordpress.wp_api import load_wp_api_one_post
@after_response.enabledef load_story(request, wp_post_id): # let WP REST API catch up time.sleep(1) try: load_wp_api_one_post(settings.WP_SITE_ID, wp_post_id) except: logger.exception("Fail! wp_post_id=%s", wp_post_id)
Webhook continued
django-wordpress-rest
This code is here:https://github.com/observermedia/django-wordpress-rest
Contact me to contribute if interested!(Or if you find bugs…)