Preparing for the unthinkable Introduction to MongoDB · Introduction to MongoDB Querying the case for document databases PHP From the Inside Out Learn how PHP’s guts fit together

Introduction to MongoDBQuerying the case for document databases

PHP From the Inside OutLearn how PHP’s guts fit together in action

Life as a WordPress plugin devWhat it takes to build a chart-topping add-on

Published by S&S Media Group

www.webandphp.com

ALsO In tHIs Issue

June 2013Issue 15

HtM5 now: Responsive Web Designby Frédéric Harper

Preparing for the unthinkableby Sebastian Bergmann

Data Modeling 103by Cory Isaacson

©iS

tock

pho

to.c

om/t

eeki

d

Contents

ContentsLetter from the Editor

Everyone is a newbie once, right? All of us, at some point in our lives, had to work out what these weird ‘variables’ and ‘functions’ do. Even with years and years of experience, a weird new trend or tech-nology might come along and leave us floundering.

In fact, sometimes even the biggest names in the tech space can find themselves the newbies once again! Windows may be on most PCs, but Microsoft haven’t had much luck in the smartphone and tablet markets: Windows Phone and Windows 8 still haven’t quite hit the mark, even after years of R&D and billion-dollar marketing campaigns.

Here at Web & PHP Magazine, we try to make sure to include articles on not only the advanced stuff, but beginners’ content, too. Maybe you know about re-sponsive design already, but if you haven’t used media queries before, you should go read Frédéric Harper’s article right now! It’s the first of a new series called “HTML5 Now”, with more advanced stuff coming later.

What else have we got in store this month? First up is an introduction to MongoDB, a NoSQL document database without a schema. If you’re been wonder-ing what all the fuss around NoSQL is about, this is the article for you – and it’s written by Derick Rethans, who coded the PHP driver for MongoDB itself.

Breaking up the tutorials is something a bit different. If you’re reading this magazine, you probably know quite a bit about the PHP language already – but how much do you know about PHP the program? On page 12, we take a peek under the hood with project con-tributor Julien Pauli, who guides us through the differ-ent components of PHP.

Last in this issue (but certainly not least) is a set of interviews with the creators of some of WordPress’s most popular paid plugins. When your labour of love gets downloaded hundreds of thousands of times, how do you cope with the sudden popularity? The de-velopers we spoke to had to learn new skills fast – and for some of them, it’s paid off big time.

And let’s not forget our regular columnists! Sebas-tian Bergmann has written about why you should worry about a “race condition” in your applications, and Cory Isaacson has continued expanding his “Angry Shards” example – all the while trying to keep it normalized.

So, something for everyone I hope! If you have any thoughts on the issue, my address is [email protected] – I’m always up for a chat, and I’d love to hear what you think. Don’t be too harsh though, I’m still a newbie!

Elliot Bentley, Editor

New beginnings MongoDBIntroduction to Document Databases with MongoDB 3Derick Rethans

HTML5HTML5 Now: Responsive Web Design 6Frédéric Harper

PHPPHP From the Inside Out 12Julien Pauli

ColumnPreparing for the Unthinkable 17Sebastian Bergmann

ColumnData Modeling 103 20Cory Isaacson

CareersLife as a WordPress plugin developer 23Interview with Duane Storey, Joe Dolson, Mikko Saari and Michael De Wildt

www.webandphp.com Web & PHP Magazine 6.13 | 2

NoSQL MongoDB

by Derick Rethans

By now most you will probably have heard of the term NoSQL. It’s a vague term that covers a lot of different types of database engines. The main classes of NoSQL databases are key/value stores, column databases, graph databases and document databases. Examples of a key/value stores are memcache [1] or Redis [2], where data can only be stored and retrieved through a specific key. Column databases, such as Cassan-dra [3] and Hadoop [4], are great for processing large amounts of data. Graph databases such as Neo4j [5] and OrientDB [6] model the relations between entities. Apache CouchDB [7] and MongoDB [8] belong to the last category, Document databases. We will be look-ing extensively at MongoDB in this article.

In a document database such as MongoDB the small-est unit is a document. In MongoDB, documents are stored in a collection, which in turn make up a database. Document are analogous to rows in a SQL table, but

there is one big difference: not every document needs to have the same structure – each of them can have different fields and that is a very useful feature in many situations. Another feature of MongoDB is that fields in a document can contain arrays and or sub-documents (sometimes called nested or embedded documents).

MongoDB’s StrengthsSupporting a different set of fields for each docu-ment in a collection is one of MongoDB’s features. It allows you to store similar data, but with different properties in the same collection. A good example of this is storing real (not MongoDB) documents in a way that is beneficial for a Content Management System (CMS). The CMS might want to store articles, which have certain properties (e.g. author, tags, and body), but also related books, which ore the periodical’s ISSN number in lieu of an ISBN number. In a relational da-tabase there are vhave additional properties such as their ISBN number, but no body field. An article may

NoSQL? No problem!

Introduction to Document Databases with MongoDB When should you drop your relational database for a shiny NoSQL alternative? Derick Rethans queries the case for document databases

Imag

e lic

ense

d b

y In

gram

Imag

e


MongoDBNoSQL

need to store the periodical’s ISSN number in lieu of an ISBN number. In a relational database there are various ways to solve this. Most frequently it is either solved by having a table per object “class” (article or book) or coming up with a scheme that stores object’s proper-ties in linked tables (for example through the EAV pat-tern). In MongoDB you would simply store the article and book with the fields they need, using the structure shown in Listing 1.

Even though the two documents represent different classes of objects, you can still construct a query that looks for all the items by an author, or for all the items with a specific title.

Data ModelEach document in a collection in MongoDB can look totally different, and how you structure your docu-ments is up to you. MongoDB doesn’t enforce a schema, but your application still should. Although MongoDB is generally very fast, the way how you structure and index your documents and collections has a big influence on the performance of your ap-plication. While designing your schema you should focus more on how the data is inserted, updated and queried and less on how the data is structured. If sometimes you need to denormalise your data, then that is a totally normal thing to do, even though it might look dirty at first.

Interactions Between CollectionsMongoDB makes different choices regarding function-ality and scaling than relational databases. MongoDB is very easy to scale through replication and sharding, but it misses out on features like joins and transac-tions because of this. Operations in MongoDB are only atomic per single document, and only operate on one collection. Not allowing operations between col-lections (joins) sounds like a real issue, but with the

support of arrays and sub-documents this is actu-ally in most cases not a problem. Let’s have a look at the following example:

Take an application where we store image (meta) data and tags that go with those images. In a relational database you would store that in three different tables (Tables 1, 2 and 3).

And queries for both meta-data and the tags for the bunny (id = 2) would be those in Listing 2.

This is quite complex as you can see. There are three tables, and two queries involved. In MongoDB, you might store the same data as in Listing 3.

To provide the same results as with the two SQL queries above, you would run in the MongoDB shell:

db.Images.find( { _id: 2 } );

Listing 2SELECT * FROM Images WHERE id = 2

SELECT value FROM ImageTags LEFT JOIN Tags ON (Tags.id = ImageTags.tag_id) WHERE ImageTags.image_id = 2

Listing 3{ _id: 1, filename: 'cow.jpg', mimetype: 'image/jpg', size: 9123, tags: [ 'animal', 'tasty' ]},

{ _id: 2, filename: 'bunny.png', mimetype: 'image/png', size: 8192, tags: [ 'animal', 'cute' ]}

image_id tag_id

1 1

1 3

2 1

2 2

Table 3: “ImageTags” table in a typical relational database

id value

1 animal

2 cute

3 tasty

Table 2: “Tags” table in a typical relational database

Listing 1{ _id: ObjectId("51156a1e056d6f966f268f81"), type: "Article", author: "Derick Rethans", title: "Introduction to Document Databases with MongoDB", date: ISODate("2013-04-24T16:26:31.911Z"), body: "This arti..."},{ _id: ObjectId("51156a1e056d6f966f268f82"), type: "Book", author: "Derick Rethans", title: "php|architect's Guide to Date and Time Programming with PHP", isbn: "978-0-9738621-5-7"}

id filename mimetype size

1 cow.jpg image/jpg 9123

2 bunny.png image/png 8192

Table 1: “Images” table in a typical relational database


MongoDBNoSQL

And on top of that, you have all the data right in one place ready for display. Most examples for MongoDB will show your documents as JSON documents. This is not how MongoDB stores it internally, but it is a good representation of how MongoDB deals with doc-uments. For use within PHP, you would convert both objects and arrays to PHP arrays. The above can be translated to PHP like so:

$doc1 = array( '_id' => 1, 'filename' => 'cow.jpg', 'mimetype' => 'image/jpg', 'size' => 9123, 'tags' => array( 'animal', 'tasty' )},

Or if you use PHP 5.4 you can use the following:

$doc1 = [ '_id' => 1, 'filename' => 'cow.jpg', 'mimetype' => 'image/jpg', 'size' => 9123, 'tags' => [ 'animal', 'tasty' ]],

PHP 5.4’s short array syntax can come in quite handy when dealing with MongoDB documents with nested arrays and objects.

Getting StartedMongoDB can be downloaded for free from Mon-goDB.org [9] downloads. If you are on Debian or Ubuntu, I would greatly advice to follow the specific instructions [10] with packages because they make updating easy. After downloading, please make sure that MongoDB runs by running on the command line

mongo test. This opens up a shell like interface for the test database. If that works, then you can issue com-mands in Java Script syntax such as:

db.persons.insert( { 'name': 'Derick Rethans', 'twitter': 'derickr' } );db.persons.find( { 'twitter': 'derickr' } );

In order to use MongoDB from PHP, you also need to install the PHP driver for MongoDB [11]. In most situa-tions you should be able to do so by running:

pecl install mongo

Please refer to the PECL manual for further installation instructions. Analogous to the previous example on the shell in PHP we would do this (Listing 4).

The PHP documentation also includes a section for working with the MongoDB driver, including a tuto-rial [12]. A handy cheat sheet [13] gives you a quick overview on how to map SQL queries to the Mon-goDB query syntax.

Closing WordsMongoDB is not a straight replacement for your re-lational database. Questions such as “How do I con-vert my relational database to MongoDB?” make little sense as such a different approach is required to write applications with MongoDB. That doesn’t mean that MongoDB is not a general purpose database – it can replace a relational database in almost every situation. You just need to approach it differently, and when you do so you should find working with MongoDB a breeze. Try it out, and stay tuned for future articles on more of its features!

Listing 4

<?php

$m = new MongoClient;

$db = $m->test;

$col = $db->persons;

$col->insert( array( 'name' => 'Derick Rethans', 'twitter' => 'derickr' ) );

foreach ( $col->find( array( 'twitter' => 'derickr' ) ) as $record )

{

var_dump( $record );

}

Derick Rethans has contributed in a number of ways to the PHP project, including the Xdebug debugging tool, and various exten-sions and additions. He’s a frequent lecturer at conferences, the author of php/architect’s Guide to Date and Time Programming, and the co-author of PHP 5 Power Programming. He is now work-

ing at 10gen to work on the PHP driver for MongoDB.

References

[1] http://en.wikipedia.org/wiki/Memcache

[2] http://en.wikipedia.org/wiki/Redis

[3] http://en.wikipedia.org/wiki/Cassandra_%28database%29

[4] http://en.wikipedia.org/wiki/Hadoop

[5] http://en.wikipedia.org/wiki/Neo4J

[6] http://en.wikipedia.org/wiki/OrientDB

[7] http://en.wikipedia.org/wiki/CouchDB

[8] http://mongodb.org/

[9] http://mongodb.org/

[10] http://docs.mongodb.org/manual/tutorial/install-mongodb-on-debian-or-ubuntu-linux/

[11] http://pecl.php.net/mongo

[12] http://php.net/manual/en/mongo.tutorial.php

[13] http://www.php.net/manual/en/mongo.sqltomongo.php


Responsive design HTML5

by Frédéric Harper

The problem: How often have you visited a website on your smartphone, only to get a really bad experience? You have to scroll up, down, right, left, zoom in, zoom out, struggle to click on the links ... It’s clear that the site just isn’t designed with your mobile device in mind (the site in Figure 1).

Maybe you received an email from a friend linking to an article you have to read – but the site was format-ted for a mobile device (probably a http: //m.s ite.com url), and you open it on your 24 inch desktop monitor (Figure 2). Again, a really bad experience for the user.

In both cases, it’s the developer’s job to make it bet-ter, and to provide a good experience for users on all devices.

Responsive Web DesignJust over three years ago, a guy name Ethan Mar-cotte [1] wrote a blog post about the concept of Responsive Web Design on A List Apart [2]. The phi-losophy was simple: let’s design our sites to respond and adapt it any screen size. It was not a new idea, but – as far as I know – was the first time that the concept was presented as a whole, and how it was also easier than ever for developers to achieve using new technologies.

This new philosophy is all about thinking of user’s needs instead of our own. If we adapt our sites to the capabilities of different devices, rather than their configurations, it will help future-proof our sites. By creating responsive sites, we can have one version providing a great experience to each user, whether

they are on a smartphone, a desktop monitor or even on their gaming console. To make it a reality, you’ll need three important elements:

1. A flexible, grid-based layout2. Flexible images and media3. CSS3 Media queries

Flexible, Grid-based layoutThe first thing you need to do to make your site re-sponsive is to change your fixed-width layout to a flexible one. Instead of using pixels, you need to con-vert those to em units or percentages using this little formula:

TARGET / CONTEXT = RESULT

What does that mean? That you take the size in pixels of the element (or the size you would like it to be dis-played on your actual screen), and divide by the size in pixels of its container (or the size you would like it to be displayed on your actual screen also): that will give you the equivalent size in percentage for your element. Let’s see a more concrete example with this actual CSS code:

#page { width: 800px; }

#page .news { width: 166px;}

If we want to change the .news element inside of the #page one, we’ll take the target of 166 pixels, divide it by the context, the div containing the .news elements,

Looking good on any screen

HTML5 Now: Responsive Web Design The web has changed: it’s no longer just about making your website fit only on a desktop screen. Our sites are now viewed on desktops, laptops, tablets, smartphones, game consoles, e-readers and even TVs. As developers, we can’t ignore how users view the web today ...


HTML5Responsive design

more CSS code, we know that our base font-size is 100 % of the body element, and the standard is usually of 16px. We’ll do the math, and get 1.5 from 24 divide by 16. For the sec-ond element, it will be easier as we have a clear con-text of 24px from the h1:

h1 { font-size: 1.5em; /* 24/16 */ }

h1 a { font-size: 0.4583333333333333em; /* 11/24 */}

Yes, the second font-size value has far too many deci-mal points, but don’t touch it: the browser will do the work, and it’s a best practice to leave these numbers as it’s the nearest result to what we are looking for. At the end, the browsers will calculate the result in pixels to display them in the browser, so don’t bother to round the number – they can handle it!

Did you see my comments on the two new CSS files? It’s not just for the learning purpose, but also for me, as the person who built the site. If I want to change the size of an element, my TARGET or CONTEXT will

of 800px, and we’ll get our result of 0.2075. To get the percentage, of course, we’ll multiply 0.2075 by 100 to get 20.75 %.

For of #page element, it’s a little more tricky. His context is the body itself, and we don’t have any size set in our CSS. In this case you may want to see if the size of 800 pixels makes sense in your actual browser window, and use your browser size of the context ele-ment (or, like Ethan in his book Responsive Web De-sign [3], do it by trial and error). Let’s use a base size of 1024, so we’ll do 800 / 1024 = 78.125 % . All these hard calculations will give us this new CSS code:

#page { width: 20.75%; /* 800/1024 */}

#page .news { width: 78.125%; / 166/800 */}

We also need to think about font-size, which should respond to different screen sizes. In that case, we’ll take the same approach, and our amazingly long for-mula of TARGET / CONTEXT = RESULT to create their em counterparts. Let’s try with this actual CSS code:

h1 { font-size: 24px; }

h1 a { font-size: 11px;}

In that case, the first one will be a little bit tricky. As stated, we would like to change the 24 pixels of h1 to his em counterpart. In that case, since we have no

Figure 1: A website designed for desktop that doesn’t provide a good experience on a smart-phone device

Figure 2: A mobile version of a website viewed on a desktop browser

Figure 3: A website viewed with a large desktop monitor



change, and in that case, I’ll need to calculate again the RESULT – and trust me, it will save you some head-aches to have those numbers within easy reach.

Flexible Images, and MediaThis is probably the easiest part of making a site respon-sive. With media element like images, or videos, you just have to set the maximum width to 100 % like this:

img, video { max-width: 100%;}

This rule means that no <img> or <video> elements will be any larger than their containing element. So, for example, if the <div> that contains the company photo is displayed on a larger screen than your own, people will be able to see the image in its full width. When people access your site on a smartphone (smartphone vs desktop examplestend to make the most sense), they will see a smaller version.

In terms of code this is an easy solution, but that leaves us with a problem: what happens if that pic-ture was taken with a high-end latest DSLR camera that takes massive 20MB pictures? It won’t be a

problem for fast Internet connections, but let’s return to our smartphone situation: of course the image will display correctly on the phone, but users will have to download the full super-high-quality image. On a cel-lular data plan, that will not only take a while but cost a lot too! It’s a known limitation of today;s responsive sites, but the W3C (World Wide Web Consortium, the organisation behind the HTML5 standard) is working on a new, responsive <picture> element [4]. I won’t go into any further depth with this one as it’s still in draft, and not quite implemented in major browsers yet, but it’s good to know that it’s on its way. For now, one alternative is to use JavaScript equivalents, like the Responsive Images script from the Filament Group [5].

Media queriesLast, but not least, are media queries. CSS3 Media Queries provide developers with the ability to scope a style sheet to a set a precise device capability. In other terms, it allows for the possibility to do some kind of if statements in your style sheets. Why would we need to do this? After all, we the size of our elements al-ready follow the size of the screen (remember the ems and percentages?). However, even if we have a flex-ible layout, at particularly small sizes it simply won’t work. This is where we need to use media queries to clean things up. The syntax is simple: select a type of media (mostly screen or print), combined with one of the supported properties like:

Figure 4: Top of the Figure 3 responsive website viewed with a smartphone

Figure 5: Middle of the Figure 3 responsive website viewed with a smartphone

Figure 6: Bottom of the Figure 3 responsive website viewed with a smartphone

• min/max-width • min/max-height • device-width • device-height • orientation • aspect-ratio

• device-aspect-ratio • color • color-index • monochrome • resolution



Figure 7: A full screen view of Little Pea Bakery

Figure 8: A tablet-size view of Little Pea Bakery Figure 9: A smartphone view of Little Pea Bakery

In most cases, you’ll play with min-width, max-width, min-height, max-height, and orientation. Let’s check a simple example:

@media screen and (max-width: 600px) { body { font-size: 80%; }}

This CSS means that for all screen media that have a maximum width of 600 pixels, the font-size will be 80 % of the original size. You can also do a combination of different properties like:

@media screen and (min-width: 320px) and (max-width: 480px) { #page { color: #FFFF00; }}

In this example, the style enclosed in this media queries (a text color of blue) will execute when the media is rendered on a screen width between 320 pixels and 480 pixels. So you can do many combinations of these properties to get different results:

• Keep in mind also that CSS3 Media Queries can be imported @import url(mq.css) only screen and (max-width: 600px) • or used as an inline element <link rel="stylesheet" media="only screen and (max-width: 800px)" href="mq.css">

If you want to see a more complete example, go to the Little Pea Bakery demo at http://stunningcss3.com/code/bakery/ (or see the screenshots in Figures 7-9). Check out the full-size layout, resize your browser to see how CSS is rendered, and view the source to under-stand how it was done (please note that the CSS has been added in the HTML code for demo purpose, but a good practice is to separate those in two different files).


www.webandphp.com


Making the world a better placeThese three ingredients for a responsive design give you the power, as a developer, to make your site re-spond to the user's needs. It will provide a better ex-perience than a fixed-width site, but there are some other questions you need to ask yourself before going down this new path. Do I need to start mobile first? Should all websites use Responsive Web Design? Do I need every visual element on every device? You’ll also need to think about old browsers that doesn’t support media queries, maybe by using a polyfill like the css3-mediaqueries-js JavaScript library [6].

In the end, whether you choose to make site respon-sive or not, never forget that, as web developers, our goal is to make the Web a better place to live.

ResourcesHere are a few more helpful resources:

• Luke Wrobleski (father of Mobile First) multi-device layout patterns [7]

• 320 and up: a Responsive Web Design boiletplate [8] • Stunning CSS3: a good book on CSS3 [9] • Media Queries: a collection of site using media

queries (for inspiration) [10] • Screenfly: a little tool to test your site on different

devices (Figure 10) [11]

References

[1] http://ethanmarcotte.com/

[2] http://www.alistapart.com/articles/responsive-web-design/

[3] http://www.abookapart.com/products/responsive-web-design

[4] http://picture.responsiveimages.org/

[5] http://filamentgroup.com/examples/responsive-images/

[6] http://code.google.com/p/css3-mediaqueries-js/

[7] http://www.lukew.com/ff/entry.asp?1514

[8] http://stuffandnonsense.co.uk/projects/320andup/

[9] http://www.stunningcss3.com/index.php

[10] http://mediaqueri.es/

[11] https://quirktools.com/screenfly/

Figure 10: Using Screenfly to test my responsive site, outofcomfortzone.net, on different devices easily

Frédéric Harper has been building software for more than a decade. Living in Montreal, he is working as a Technical Evangelist at Mi-crosoft, sharing his passion about application development. Lead-ing the Make Web Not War initiative, Open Source, Open Standard, and Interoperability are the pillars of his journey at the Empire. Ex-

perienced speaker, t-shirts wearer, long-time blogger, passionate hugger, and HTML5 lover, Fred has multiple personalities. Always conscious about the im-portance of unicorns, and gnomes, you can read about these topics, and oth-er thoughts at outofcomfortzone.net.

Available and on sale for

$ 3.99Buy now at:

www.developerpress.com

Also available to buy on:

Sep

tem

ber

16

– 19,

201

3 S

an J

ose,

CA

@webandphp

Core

PHPhtml5

css3WordPreSs

Drupal

UXMobile

Frameworks

NodeIjs

www.webandphp.com

SESSIONS . WORKSHOPS . KEYNOTES

Ho! Let’s Code!

Hey!

Internals PHP

by Julien Pauli

PHP is a complex machine written using the C language (C99). Nowadays, the basic PHP distribution (including default extensions) consists of about 700,000 lines of C and 30,000 lines of PHP for testing. Yes, PHP is a tested program of which tests are written themselves in PHP!

As with any program, it has been sliced into several distinct parts which each have individual responsibili-ties and talk to each other. PHP is an open source pro-ject, so everybody is free to download its source code and study it, which I have been doing for a few years ago now. It’s a very interesting and challenging project that is driven by its user community and a number of

contributors around the world. You may study its code by looking at the GitHub mirror [1].

This overview outlines how PHP is organized inter-nally, so that you may have better understanding of how things work. While reading, you should keep in mind that PHP has been designed to optimise memo-ry consumption and execution speed, and to be web oriented, scalable and easy to learn. Sometimes you may notice the consequences of those decisions while reading the source code or while trying to under-stand some concepts that, on the surface, may appear strange or even downright silly.

Finally, don’t worry – we’re not about to dive deep into cumbersome C code. Instead, relax and enjoy this guided tour through the beating heart of PHP.

An overviewPHP is primarily composed of four large blocks (as il-lustrated in Figure 1):

• PHP Core • Zend Engine • Extensions • SAPI

These four blocks can each themselves be divided into smaller parts, which we’ll study later. First, however, we’ll start with SAPI.

SAPIThe Server Application Programming Interface is the entry point for reaching PHP and its organs, and is of-ten considered to be a fuzzy feature. You may already know about several different SAPIs.

One is the Command line interface (CLI) SAPI, which contains the main() C function and allows anyone to reach PHP from the command line. It parses argu-ments passed into the CLI, and can read from its Stdin

Looking under the hood

PHP From the Inside Out PHP 5.5 co-release manager Julien Pauli provides a quick tour of PHP’s guts and how they fit together in action

©iS

tock

pho

to.c

om/A

rtyF

ree


PHPInternals

and write to its StdOut/StdErr like any shell-based Unix program. Try it by using the command php -h and see what you can do with it. You can have a read of its source code on GitHub [2].

Next, we have the CGI, or Common Gateway Inter-face protocol implementation for PHP (if you’re not aware of CGI already, look it up). CGI is not very far from the CLI implementation, the main differences be-ing that the CGI SAPI can interpret the Common Gate-way Interface protocol and answers using HTTP and HTML keys. Note that the CGI SAPI can also serve as a FastCGI implementation.

The Fast Process Manager (FPM) is a design of the FastCGI protocol. FPM first appeared in PHP 5.3 and is meant to replace the FCGI SAPI, which is outdated and lacking in features. FPM provides many more op-tions than the old FCGI SAPI, such as static or dynamic pools, per-pool ini settings, per-poll environment and status control page.

Apxs2: Apache Extension 2, this is the “PHP mod-ule” for Apache2 that binds PHP engine to Apache2 one, talking to each other. It allows embedding PHP into Apache2 server.

Many more SAPI modules exist [3], these are sim-ply the best-known examples. Of course, PHP’s worldwide spread is mainly down to its widespread integration into a large number of web servers and architectures. With the SAPI providing an entry point, some of its goals are to:

• manage headers for the incoming request • manage incoming request data, such as POST or COOKIE

• write on the output layer, should it be an Apache buffer, a Unix process stdout, or something else

• start and stop the other components, ZendEngine and PHPCore

The Zend EngineThough its border with PHP Core is tight, the Zend En-gine actually uses a different open-source license to the PHP licence (though the two are compatible), and can be thought as a complete project in itself. In fact, it is – some big companies have managed to separate PHP from its engine (for example IBM and Facebook). While no trivial task, it’s actually not that hard to do, as the Zend Engine has been thought of as being some-how decoupled from PHP. The Zend Engine’s main components and goals are as follows (Figure 2 pro-vides a visual representation of these):

• Compiler: lexical analyser, syntax parser; these engines are involved in not only understanding the syntax of PHP, but also INI syntax parsing and serialized format syntax parsing; additionally they may be used by extensions such as parsing date formats, or SQL for PDO

• Executor: The biggest and most important part of the Zend Engine is the Zend virtual machine that PHP benefits from

• Memory management: The Zend Memory Man-ager (manages heap and somehow stack)

• Core features such as classes, functions and PHP variables (references, types, management)

• Thread Safety manager, when needed (though this is not really a Zend feature)

• Zend_gc : PHP’s garbage collector • Advanced C float type manager • Error and exception engines • Extensions engine • HashTables (PHP arrays) and the sorting algorithm (quicksort)

• PHP objects and iteratorsHigh level C structures (lists, linked lists, stacks, sets etc.)

• Multibyte chars for lexical analyzerFigure 2: The Zend Engine and its components

Figure 1: The four major components of PHP, and how they interact


PHPInternals

As you can see, Zend Engine really is the heart of the PHP language. It lives in the Zend/directory of the source tree, and is built independently from other PHP components such as PHPCore, SAPI or extensions, though it obviously gets linked to the final PHP binary when compiled.

The engine also car-ries the Object concept, which you should be aware of. It appeared in Zend Engine 2, which was released with PHP 5.0.

As it is a project itself, ZendEngine provides its own version number which you can find with ei-ther the zend_version() function from PHP, or witihin phpinfo() output.

PHP Core:As you may have guessed, PHP Core is all about PHP itself. Don’t confuse it with the Zend Engine layer (though it can sometimes hard to feel the border). PHP Core’s responsibilities are to:

• Manage network through the stream API • Sorting algorithms (MergeSort) • Reading arguments from the command line (getopt()) Parsing and normalizing paths (fopen)

• Output buffering system (ob_)Ticks • Logos (php and zend) • Reading icomming request parameters (passed to the SAPI layer)

• Global variables, url parsing, string management, error handlers

It may seem obvious to say, but the Core manages what the Engine and Extensions do not.

For example, the engine expects some utility func-tions to be used at different level of its work, “fopen_function”, “error_function”, “on_timeout”, and so on. These functions are written in PHPCore, and then passed to the engine for use. This way, they serve

separate roles but are tied together.

To provide another ex-ample: all SAPIS expect to be passed some parsing functions, such as default_post_reader, treat_data and input_filter. Again, these functions are imple-mented into PHPCore and

bound to the SAPI layer when PHP starts.PHPCore also carries all “php.ini” logic. The Zend

Engine can parse an ini file, but what happens to those extracted values once parsed? PHPCore registers them all into a configuration array, knows how to dis-play the entries, and calls some functions when han-dling special values from ini files, such as “extension” or “max_execution_time”.

In fact, SAPI and Zend Engine are two “projects” that expect some function pointers to be provided for the “real” work to happen. These are designed into PHP-Core by default, but anyone could use those layers by replacing the functions PHPCore provides and rewriting the other components to carry out a different function. For example, you could write a json parser that reads json from the request, parses and understands it, and you’d end up with a program like PHP – but one that would be able to understand a json request internally.

ExtensionsOnce again, it’s easy to get confused regarding the roles of PHP’s components. The Zend Engine pro-vides an extension mechanism which allows (big)

parts of code to be loaded at engine startup (or dur-ing runtime using PHP’s dl() function).

What is potentially confusing is that some parts of PHP and of the Zend Engine are provided using ex-tensions. Both “Core” and “Standard” extensions are examples of such a strange manner of thinking. Run-ning phpinfo(), you should be able to see them (but not disable them). The “Standard” extension provides features such as:

• array_ functions • str_ functions • PHP serialization • URLs and string managers • number bases, locales • crypt, base64, md5, assert, crc32, phpinfo • file system handlers and low-level stream functions

The “Core” extension is responsible for publishing some of the Zend Engine features to the user, including:

• func_get_args() • each() • interface_exists() • set_error_handler() • gc_collect_cycles()

A simple command such as php --re core will show you all this stuff. Feel free to analyze phpinfo() output as well.

And, overall, you should be aware of tons of exten-sions: json, mysqli, pdo, xml … You know ...

The extension mechanism has pushed PHP into a very extensible language. It is nowadays very easy to build an extension and tell it what you want to do (assuming you have enough knowledge of C). The mechanism provides all C structures and functions needed for a developer to start embedding any library, even one written in C++, into PHP [4].

“PHP’s lifecycle has been designed so that one process may serve multiple requests

in a total isolation mode.”


PHPInternals

Extensions, and particular Zend extensions (the oth-er “kind of” extension PHP can use) may modify fea-tures deep within the engine, such as rewriting parts of the Zend Virtual Machine, without the need to patch the source directly.

How does all this actually work?PHP has a particular lifecycle by design. It’s been made so that one process may serve multiple requests in a total isolation mode. Than means that, for the same process, the request N should know absolutely noth-ing about the N-1 request.

To achieve this, PHP’s heart has been designed so that it prepares components to treat a request, treat it (run the PHP code, produce a result), and clean all the request-bound environment, but leaving alive the more global PHP environment.

For example, you should know that when PHP is treat-ing multiple requests, it will only trigger the parsing of php.ini once for all requests. But, at the same time, PHP will trigger its parser and executor for each request.

PHP has two main scopes internally: the “module scope”, which is never cleared except when PHP ter-minates or dies, and the “request scope” which is cleared and reset for each request.

Module scopeThe module initialisation routine can be represented as in Figure 3.

This is a call graph of nearly every C function which gets called when PHP starts. The module initialization routine (called “MINIT”) must start every structure and component PHP will further need. Remember, this cycle is only run once per PHP process lifetime.

Request ScopeOnce the MINIT processing has been car-ried out, PHP is mounted into memory, and is ready to treat as many requests as you want. When a request knocks at the door (through the SAPI layer), the Request scope is entered, and what is called the “RINIT” process is triggered. This is illus-trated in Figure 4.

Here, the engine will prepare the parser and the executor, and trigger a startup() function on all loaded extensions, which also benefit from this phase.

Treatment stepWe are now ready to execute the script from the request, as outlined in Figure 5. PHP compiles and executes the script, pro-ducing – as you might expect – a result.

Figure 4: The RINIT process

Figure 3: A representation of the module initialisation routine when PHP starts Figure 5: Script execution functions


PHPInternals

Shutting downOnce the script(s) have been executed, PHP enters a phase where it has to shutdown the request scope to be ready to prepare a new one. This is called the “RSHUTDOWN”, illustrated in Figure 6.

Here, PHP clears everything related to the previous request. This allows PHP to free up some memory, although important structures prepared though the “MINIT” cycle won’t be freed up here. Those will be released when PHP shuts down completely, the so-called “MSHUTDOWN” cycle, that we won’t detail in this article.

ConclusionTo see all of the above in action, see Figure 7, which uses the Apache server as an example.

The design of PHP has never fundamentally changed throughout its 18-year history, and it doesn’t benefit from an “execution server” which keeps track of ob-jects and variables (and external service connexions) in memory from one request to other (as some lan-guages do, like Java for example). Some exceptions exist, and well-designed extensions could change that.

PHP is an “execute and forget the context” lan-guage, which is what allows it to be sovery fast. PHP is fast and memory efficient: A typical PHP process consumes about 20Mb, up to around 50Mb for the biggest ones that embed complex frameworks and classes. If you have heavier processes, chances are that you loaded several extensions which may not be well designed, or that your developers are demanding silly things from the language (such as constructing a very large array, or using file_get_contents() on a huge file). Yep, knowing how a system works helps here! PHP is not a wizard, and it can’t prevent a birdbrained user from shooting themselves in the foot.

However, PHP is clever enough to avoid parsing the same script in each request (which is time and CPU

Figure 6: The RSHUTDOWN phase

Figure 7: apache_request_cycle

consuming). OPCode caches have been designed so that the parser only parses one file once, and keeps the result in shared memory, so that future requests don’t need to trigger the parser again. A big step is therefore skipped over, allowing PHP to run even faster. PHP 5.5 will be released with an embedded and highly effi-cient OPcode cache (“OPCache”) that should comfort anyone choosing to use PHP in their web applications.

References

[1] https://github.com/php/php-src

[2] https://github.com/php/php-src/blob/master/sapi/cli/php_cli.c

[3] The complete list can be found here: https://github.com/php/php-src/tree/master/sapi/

[4] Visit http://pecl.php.net for an example of extensions.

Julien Pauli is a system programmer who works for BlaBlaCar in Paris. He’s been using PHP for a decade now. Few years ago, he decided to step down a layer, and direct his head and skills into PHP source code, to help improving it and to design extensions for spe-cific needs. He is PHP 5.5 co-release manager.


https://github.com/php/php-src/tree/master/sapi/Link

Quality Assurance Column

Every week I am where IT problems occur: sharing my experience with development teams around the world and helping them make better use of the PHP platform. When you spend so much time on the pro-verbial road, “interesting” things related to travel are bound to happen. For instance, over the years I have learned more about the IT systems of airlines than I ever wanted to know.

A while back I booked an itinerary with flights from Germany to the United States and back that caused an unusual problem. The purpose of the trip was to visit a customer in Washington, DC and so I booked CGN-MUC-IAD-FRA-QKL. After booking the itinerary on the website of my airline of choice, I called their customer hotline to upgrade one flight segment using so-called eVouchers. These are vouchers for free up-grades that you get when you renew your status with

this particular airline. They put me on a waiting list for the upgrade, and less than twenty-four hours later the upgrade was confirmed.

However, the “automated waiting list upgrade sys-tem” used award miles instead of eVouchers. So I called the hotline again and reminded them that I want-ed to use eVouchers instead of award miles. They told me that “they do not have a button” in their application to fix the problem, but there was no need to worry, as they would ask their technical support to “fix it manu-ally”. As it so happened, I had another flight booked for the very same day, so after the phone call I left for the airport.

Having arrived at the airport terminal, I received an email from my customer informing me that there had been a misunderstanding and that they needed me in Atlanta, not Washington, DC. Since I had sometime to

How an airline’s IT system was thwarted by a “race condition”

BioSebastian Bergmann is a mastermind of PHP development and PHP quality as-surance. He has contributed instrumen-tally to transforming PHP into a reliable platform for large-scale, mission-critical projects. Companies and PHP developers around the world benefit from the tools that he has written.

Preparing for the Unthinkable by Sebastian Bergmann, thePHP.cc, Germany

kill before boarding my flight, I approached the airline’s ground staff and explained that I would like to cancel the upgrade as well as the whole itinerary and have the miles and the money refunded, respectively. Un-fortunately, they told me that they could only cancel the upgrade and refund the award miles used to buy it. To cancel the itinerary itself I would have to either call the hotline or use the website. After cancelling both the upgrade and the itinerary (via the website) the award miles for the upgrade were refunded ... and the eVouchers were deducted.

The actions “refund the award miles and use eVouchers instead”, “cancel the upgrade”, and “can-cel the whole itinerary” that I requested were per-formed asynchronously and out of sequence. This lead to something that in computer science is known as a “race condition”. These race conditions occur when separate processes read and write the same data.

While implementing your test strategy, no matter if you are working on technology-focused unit tests that exercise a single unit of code in isolation from its col-laborators to ensure that the code works correctly; or on a business-focused test that exercises the whole software system to ensure that you are building what


www.webandphp.com

ColumnQuality Assurance

you are supposed to build, you usually start by testing what is called the “happy path”: using a well-defined input, (a part of) the software is executed without exception and producing an expected output.

Once the happy path has been covered, you shift your focus to the things you know can go wrong. For example, the system might raise an exception for a specific input type, so you invoke it with that input and ensure that the expected exception is raised. In most cases this is fairly easy, as the code communicates clearly what it expects to go wrong and how it deals with these situations. It gets complicated, though, when it comes to error conditions that the code does not check for.

I would like to leave you with a quote by James Bach: “Testing is an infinite process of comparing the invisible to the ambiguous in order to avoid the un-thinkable happening to the anonymous.” When you design your application, and plan the testing effort for it, you need to think of weird edge cases, for instance concurrent actions such as “refund the award miles and use eVouch-ers instead”, “cancel the upgrade”, and “cancel the whole itinerary”. The soft-ware does not necessarily have to be able to deal with these edge cases in an automated way. But it must record an audit trail of all actions taken, so that it is possible to resolve such issues manually if need be.

“When you design your application and plan its tests, you need to think of weird edge cases.”

Buy now at: www.developerpress.com

Also available to buy on:

All available for

under $6

Your brand new

one-stop-

digital-bookshop!

Big Data Column

For the past several entries in my column I have cov-ered the fundamentals of practical data modeling techniques. Each of the basic principles of Entity Re-lationship Modeling has been covered so far, includ-ing the idea that all data is relational. This month I will delve into more advanced normalization and de-normalization concepts, extending the Angry Shards game data model example.

The Angry Shards Game DatabaseIn the previous article, I built the start of the Angry Shards game data model, a demonstration database used at CodeFutures [1]. The first few normalized ta-bles – shown in Figure 1 – were built out, including the player, game, and player_game entities.

I’m going to be adding some new cool features to the model, to support a richer game experience for the user – and of course to demonstrate some new data modeling techniques.

Adding New Features to a Data Model Let’s add a new feature to the model: support for mul-tiple rounds in the game. This way, as players get bet-ter at the game and advance levels, we can track how well they do in each round. To make this more interest-ing for the player and to encourage competition, let’s record the highest score the player achieves, as well as the highest level obtained.

For our initial attempt at this, we will add some col-umns to the player_game table. To support up to 3

rounds for each game, let’s add three new columns to the player_game table: game_round_score_1, game_round_score _2, and game_round_score_3. Each of these columns will hold the resulting score for that particular round of the game. Similarly, to record the highest level and score, we will add high_level and high_score columns to the player_game table. The new model is shown in Figure 2.

I’m sure you may have already noticed this is not the ideal approach, as it causes several problems and limitations in our data model:

• The game is limited to 3 levels, unless we continue adding a new column each time another level is designed in. This is a hassle, and makes the model very inflexible.

Keeping our ‘Angry Shards’ database normalized while adding complexity

Data Modeling 103

by Cory Isaacson

BioCory Isaacson is CEO / CTO of CodeFutures Corpora-tion. Cory has authored numerous articles in a vari-ety of publications including SOA Magazine, Data-base Trends and Applications, and recently authored the book Software Pipelines and SOA. Cory has more than twenty years experience with advanced software architectures, and has worked with many of the world’s brightest innovators in the field of high-performance computing. Cory has spoken at hun-dreds of public events and seminars, and assisting numerous organizations address the real-world chal-lenges of application performance and scalability. In his prior position as president of Rogue Wave Soft-ware, he actively led the company back to a position of profitable growth, culminating in a successful acquisition by a leading private equity firm. Cory can be reached at: [email protected].


ColumnBig Data

• It’s not easy to track any other information about each round of play without adding multiple columns for each piece of data needed. For example, if we want to know how long it took the player to play a particular round, this approach would acquire 3 additional columns: game_round_time_1, game_round_time_2, and game_round_time_3. Therefore, as we enrich the game experience, the model will become more and more unwieldy and cluttered – making it harder for the development team to un-derstand as well.

• Recording the high_score and high_level on the player_game table is just plan incorrect, as it direct-ly violates the rules of data model normalization.

The reason? Because the high_score and high_lev-el are not related to player_game, but rather belong with player table, since a player can have at most one and only one value for these attributes. Be-sides breaking theoretical rules, there is a far more practical side to this common error: each time you want to record the high_score and high_level val-ues (at the end of each game), the application must update every player_game row for the player. This is incredibly inefficient as an active player could have 100s of player_game rows, resulting in a large number of updates to the database (and updates are almost always the least efficient operation in a production database).

Given that our objective is a highly flexible model that will support the life of the application and be as easy to work with as possible, we will apply some simple normalization techniques to fix it.

The first step is to move the high_score and high_level columns to the player table. This associates these columns with the table to which they are truly related, as they can only occur once and only once per player instance.

The next step is to normalize the repeating fields for recording the scores for each round or level. Since the actual relationship between a player_game to a round or level of play is one-to-many, we need to add a new table: player_game_round. We will add the following attributes to the table, resulting in the model shown in Figure 3:

• player_game_round_id: Sequential ID, matching our standard for other tables.

• status_code: The status of the round, a numeric code value representing in-progress, complete, abandoned etc.

• round_play_time: The length of time it took the player to complete the round.

• round_score: The score that the player achieved for the round.

• round_level: The level of the game that the round was for.

• player_game_id: The foreign key to the immedi-ate parent table (player_game). This allows us to express the relationship of a given player_game instance to its player_game_round instances.

You can see that, after applying our normalization tech-niques, this version is much improved. Now we can flexibly add as many levels to the game as we like, and track various attributes about how a player does while playing those levels. In addition, we can track the Figure 1: The “Angry Shards” game data model, as we left it last week Figure 2: The model with additional columns for high scores


ColumnBig Data

high_score and high_level for each player – very useful information that we can provide to the end user.

It should be obvious that with a little work and thought, database normalization techniques can dra-matically improve your data model, making it more flexible, efficient, and much easier to understand by your development group.

De-normalize your model?!Now that we have covered database normalization, what about de-normalization? Why would you want to do this? When do you use it? These are good questions, as at this point in the process you will

have likely spent a lot of time and effort normalizing your model.

While database normalization techniques are ex-tremely workable, and provide the backbone of data modeling, the truth is that in some cases a fully nor-malized model can become unwieldy or inefficient. Based on this, after I finish normalizing a data model, I then de-normalize it as required using the following rule as a guideline: “De-normalize a data model for convenience and performance.”

By de-normalize, we mean breaking some of the da-tabase normalization rules, typically duplicating certain data elements in the structure.

Most of the time de-normalization occurs during the development process, when application developers run into functions that are difficult to implement in the fully normalized structure, or when performance bot-tlenecks are encountered. By selectively de-normaliz-ing your model where needed, you can speed up the development process and often make your database perform dramatically faster [2]. Some common ways you can de-normalize your data model are:

• Create a special table that joins selected attributes together, providing easy and fast access. This can eliminate some expensive joins for frequently ac-cessed data, making the job of your developers easier at the same time.

• Resolve challenging relationship structures, such as a social network graph, and links between entities.

• Store pre-computed values for fast access to spe-cific dashboard or statistical information.

These are just a few of the ways you can use de-normalization, providing you with a preliminary under-standing of the topic. Next month I will detail some actual de-normalization examples to provide more in-formation on this topic.

Wrapping it upIn this article, I covered database normalization tech-niques, and also when to selectively use de-normali-zation. These approaches make a data model far more meaningful, easier to work with, and increase flexibil-ity – plus help to maximize performance. Practice in using these techniquees is the best way to learn them inside-and-out – soon you will become a fluent data-modeler, able to apply these concepts to virtually any DBMS. With this knowledge, you’ll be well prepared to deal with scaling your data, maintaining a solid struc-ture that can work well in distributed environments.

Figure 3: The model, with “player_game_round” details moved to a separate table

References

[1] The Angry Shards game, data model and graphics are copyright © 2012-2013 by CodeFutures Corporation and is used with its permission.

[2] Another reason to de-normalize that I will cover in the future is when you are scaling your database – often de-normalizing just a few tables or data attributes can make an enormous difference in your ability to scale your database across a distributed cluster of servers. Much to come on database scalability in future articles – that is of course the main purpose of the column in the first place.


WordPress plugins

“It was sort of a natural evolution” Duane Storey and Dale Mugford run BraveNewCode, the devel-opment studio behind mobile site plugin WP-touch [1]. The free version has been downloaded over four million times from the WordPress pl-ugin repository, and BraveNewCode now has five people working full-time on development and support of WPtouch Pro.

Web & PHP Magazine: How long have you been working with WordPress?Duane Storey: It’s actually interesting – I was in the Vancouver area. I’m an engineer by education, in the VoIP field, and I actually had a bit of an accident and was injured. And as part of the cathartic, sort of, pro-cess, I set up a WordPress blog, and I started writing. It was almost like an online diary or whatever, just to sort of talk to people and let people know how I was feeling, and I went through some surgeries and recov-ery. But because I’m an engineer and I have sort of a software background, I started immediately tinkering around with WordPress and eventually started writing my own plugins and whatnot. So that sort of naturally snowballed into where I am today, with full-time work-ing on WordPress plugins.

WPM: Why did you start developing WPtouch?Storey: It was actually a proof-of-concept we did a long time ago for a famous Canadian musician called Matthew Good. At the time, we were working on his website, doing client work, and Dale and myself thought it would be kind of cool to try and make an iPhone version of the site, so you could access it from all of the sites. Because, y’know, it would be cool to see tour dates and stuff like that, all the sort of musi-cian content on his site.

So we hacked that out in about a week, and we launched it on his site, and it became really popular with people – because the iPhone was really new at the time, this was like five years ago. People really took off with it and what we were doing, so we decided to change it a bit, make it less musician-centric, and then

Add-ons to their day jobs, or something more?

Life as a WordPress plugin developer 24/7 support, demanding users and ever-changing APIs: life as a developer of a popular WordPress plugin is tough, but for those who rise to challenge can also be life-changing. We spoke to four top plugin developers to under-stand what it takes to build a chart-topping add-on.

we put a version of that in the WordPress repository. Since then, I think we’re up to 3 million downloads on that one. [laughs]

WPM: When did you decide to create WPtouch Pro?Storey: It’s funny, because it was sort of a natural evolution. I was telling someone else about this the other day – it became so popular, it became unsustain-able, and we had people emailing us all the time. And we were just moonlighting, doing WordPress stuff – y’know, Dale and I both had day jobs. And I’d get phone calls on phone, people asking for my support ... so we really did hit a point where we really did have to make some money on it, or give it up. And we didn’t really want to give up on our vision of WPtouch, what we wanted it to be. And we were really worried that if we sort of let eve-rybody attack it, it would become this least com-mon denominator sort of product.

Careers


CareersWordPress plugins

So eventually we just hit a point where we had to make some money off it, so the idea was to make a commercial version which had a few more bells and whistles, and also came with support. We launched that almost three years ago now.

WPM: Did you receive any backlash for that?Storey: The overall reception was positive, but there’s definitely a few people who were negative, and unfortunately they’re usually the ones who are the loudest. The main complaint at the time wasn’t [about] what we were trying to do – it was that people were confused about commercialising GPL products. I had this one email thread in particular where some-one emailed me, and said it was illegal, and they were gonna get legal people involved, because we can’t sell it, and he basically – y’know, if you read between the lines, he wanted me to send him a free copy! But it just went back and forth, and I’m like, no, we’re allowed to do it. And I pointed him at a website that says you’re allowed to that, and he said, well, that’s just a website, I’m sure lots of websites say that. And I’m like, yeah, but that’s the GNU GPL website, that’s where the license came from, so they would know! [laughs]

And he ended up CC’ing Matt Mullenweg on that email thread, and then Matt got involved in that and – you obviously know Matt Mullenweg, he’s the guy who wrote WordPress or whatever – and he backed us up and said, they’re allowed to do that, and it’s fine.

WPM: How did you go about marketing the plugin?Storey: So, it definitely helps – that freemium model. But most of the time there’s just enough websites pro-moting us, and enough peers that sort of talk about us. We don’t really spend lots like other people, on adver-tising and things like that.

WPM: Do you think it helps to be on the top charts of the plugins repository?Storey: Well! I can’t really think to that, I’d like to be-lieve that it’s because it’s a really great product, and we’ve put a lot of polish into it. I mean obviously timing with the free version has something to do with it, because we were one of the first – I think we were really the first plugin that supported all the touch features in WebKit and things like that. But we have spent an enormous amount of time on that, and on the free version. We get reviews all the time saying that the Pro version is the nicest plugin people have used for WordPress, and we’re really proud of it, and we spend a lot of time on that last 10 % when we work on it, to make sure it’s really pol-ished.

I think we’ve developed WPtouch primarily – when we started, it was for ourselves. It’s always been a product that we’ve made because we want to use it ourselves. So we’re constantly testing it ourselves, and sort of evolving the product based on how we’re using our phones. I mean, we all have iPhones and iPads and iPad Minis and things.

WPM: How have you managed scaled up to a team of five?Storey: We hit a point where it was Dale and myself right after the launch, and we were doing all the cus-tomer support ourselves. And after a few months of sales, it quickly became clear that we couldn’t keep doing that, because we were handling the support, but we weren’t moving the product forward. So it just hap-

“Most of the time there’s just enough websites promoting us, and enough peers that sort of talk about us. We don’t really

spend lots on advertising.”

pened that there was a guy in our forums, Emilio, and he was actually a customer. And he was going round answering all these questions for people anyways. So we just called him up one day and said, hey, do you want a job? and he said “sure”. So he’s been running

support for us for a cou-ple of years in Los Ange-les, and when we got a little bit bigger, we hired another person, Diane, to help out. It’s always just been an evolution – whenever the sales have risen to a point where we think we could use anoth-er person, we’ve brought someone strategically

on. We brought on a marketing person about a year and a half ago – Joyce, she kind of handles the market-ing/media side of things. And we have Martin, who’s a full-time developer in the Hamilton area. And it’s quite likely we’ll grab a few people soon, based on some of the things coming out in the next year, year and a half.

WPM: What percentage of free users purchase the Pro version?Storey: I can’t answer that, because we don’t actu-ally know how many real instance of WPtouch Free are out there. I know downloads, but that doesn’t equate to real usage, because someone could down-load it and not use it. And I haven’t come up with a way to track that without being sort of unethical, so we haven’t bothered. But definitely, of the sales we get, about 30 % of them come from the free version. That, I know pretty much for sure.

Before we even launched a Pro version, we had a ‘donate’ link in WPtouch. And I think when we launched [the Pro version] we’d had about 1.5 million



[free] downloads, and I think we’d made a cumulative of $ 50 off of donations, right? So, I think the majority of free people aren’t inclined to pay, and that’s fine. They’re entitled to use those products without paying.

WPM: Would you consider abandoning or selling off WPtouch?Storey: Someone sent me a weird email about that last week – if we wanted to sell it – and I guess the answer is “no”. I wouldn’t say we’d ever be opposed to it, but it is essentially BraveNewCode, so we’d ba-sically be selling BraveNewCode, which we wouldn’t wanna do. And I wouldn’t want it to go to some ... – because all the people would want the traffic, and they wanna do something kinda unethical with that traffic! So, for me, if a person wanted it, they would have to continue on with the vision, and continue making a GPL product for WordPress, and continue taking it as seriously as we do. But I wouldn’t want someone to take it, and just destroy it, just to get some SEO juice out of it or something.

WPM: How has your life been changed by WPtouch?Storey: That’s a really good question, actually. I look back to before I had even got involved in WordPress, and I’d been five years in sort of a cubicle job in Van-couver. It was a good experience, but it was sort of an end-of-life for that career path, and WordPress came along at just the right time. We started with cli-ent work, and that was great, but now we’re 100 % product-based, and for me, personally, that’s meant I have a lot of freedom.

I generally work pretty hard and I work normal hours, but I’ve done a lot of travelling over the past two years because of that freedom. Like, I think I’ve been to 18 countries or something, and I usually just go for a month or two to different countries and just work from there. I mean, most of the products – that

that’s kind of the path I took was, y;know, just decided I’m going to sit down and I’m going to learn how to do this, because I think it’ll be applicable to my freelance and development business. And then it turned out to start getting more popular. And that’s when I had to start thinking about things like doing pro plugins.

I view it in some ways as I had to start thinking about it, because though it is actually – I feel it’s valuable to contribute free plugins to the respoitory, and I don’t regret doing that at all, but it is in no way profitable. For a long time I would publish the per-download earnings that I had on plugins, based on donations. And it was, y’know, a quarter of a cent per download kind of thing. Not exactly, y’know, not even paying for my develop-ment time or support time.

So, my path was more along the lines of “I have an existing plugin which is quite popular”, so I built an upograde that people could purchase. And that’s been pretty effective at supporting the development needs and processes for that plugin and support.

WPM: The free version of the plugin, WP to Twit-ter, has now had over 1.5 m downloads ... how did it become so popular?Dolson: Y’know, I was just thinking about that the other day, trying to track back in my head exactly how it became popular. It was fairly rapid. I didn’t do any marketing to speak of, but I did this – I originally wrote it because a friend of mine, Pierre Far who now works at Google, was building a URL shortener package [Cli.gs]. And he had built an API for it, and I just kind of decided, well, y’know, I’m gonna learn this API, and I’ll build something for my friend, and I’ll learn WordPress plugins, and I’ll learn the Twitter API, I’ll do all these things at once and get things going. And it made a big difference that I was creating a product that interfaced with somebody else’s new startup product that they were advertising heavily.

“The plugin was really just a stone around my neck”Joe Dolson runs a web accessibility consultancy and develops WordPress plugins on the side. His most popular creation is WP to Twitter [2], which allows tweeting of blog posts from within WordPress. Now at 1.5 million downloads, it is supported by a paid equivalent, WP-Tweets Pro.

code that’s in WPtouch Pro and PiggyPro and Word-Twit Pro, it’s cumulative of 18 countries’ worth of cod-ing, you know!

So, for me, it’s really given me the freedom to – in lots of areas of my life. It’s been a really rewarding ex-perience, seeing how people like the products and like what we’re doing.

Web & PHP Magazine: How long have you been working with WordPress?Joe Dolson: I’ve been working with WordPress since 2006 ... yeah, that sounds about right. And I started de-veloping plugins in 2008. And my biggest, best-known plugin, WP to Twitter, is actually the very first one I wrote. Which was not the plan, it’s not like I was saying “Oh, I’m just going to learn how to write a WordPress plugin and it’s going to become hugely popular”. That was really more of a surprise than everything else. So



So, in a lot of ways I was able to piggyback on the marketing that he was doing already. Because he was like “oh, this is great, my service is only a month old and I’ve got a WordPress plugin that works with it, that’s awesome!” So he really was big on promoting it. And that made a big difference. Ultimately once he took the job with Google, he had needed to get rid of a lot of his other projects, somebody else bought it and the API collapsed. So that service is no longer in the plugin, but the popularity stuck around.

WPM: What challenges did you face while develop-ing the plugin?Dolson: The biggest challenge was when Twitter decided to switch their authentication method over from basic authentication to OAuth. And that was one [decision] where I debated over whether I even wanted to do it, for a long time. Because at that time, I felt that the plugin was really just a stone around my neck, it was consuming a lot of development and support time and earning nothing, and I had a busy client load, and I was like, “Ah, I don’t know if I want to do this”.

I did decide to do it, [but] there were some rough patches ... if you look at the ratings on WP to Twit-ter, there’s a lot of great five-star ratings, but there’s a pretty good chunk of one-star ratings. And a good per-centage of those showed up during that period where I was converting to OAuth.

WPM: What made you decide to stick with the plugin? Dolson: Really, the decision came down to the fact that I had this fairly large-scale plugin, and I didn’t want to let the people using it down. I wanted to make sure it kept working. And so I just punched through and said, OK, I’m going to do this. That may have been a little crazy, but it did work.

WPM: How has your lifestyle been changed by the income?Dolson: Well, the income hasn’t had a direct impact on my lifestyle, but certainly support has. I have to be, y’know ... not 24/7, because I have to sleep! But certainly seven days a week, I’m checking support forums regularly, I’m checking email regularly, trying to actively search for anybody who’s having issues. I check Twitter and do a search on my plugin periodi-cally to see if anyone is making comments that are not directed at me.

It’s not like it’s something that consumes a huge part of my day – it’s not like I’m doing five hours a day doing support. But I have to be doing it continuously, all day long, because I need to be reactive, I need to respond quickly. So I get a request a 9 o’clock in the morning – I want to respond to it before 10, ideally. I do have to take some days off sometimes. I didn’t re-spond to very much on Saturday, because I was speak-ing at Word Camp, and I was in sessions all day, and it wasn’t really going to work. But I try to as much as I can.

Web & PHP Magazine: How long have you been working with WordPress?Mikko Saari: I think something like five or six years. I started with Movable Type, and back then WordPress – I didn’t like it at all, but at some point I made the switch over to WordPress and, well, nobody mentions Movable Type any more.

WPM: When did you start developing WordPress plugins?Saari: Well, I’ve [only] ever developed this one plugin, and it started in 2009. So, four years ago. And that’s basically because I needed it for myself. I wasn’t happy with the search in WordPress and I wasn’t happy with the available plugins to fix it, so I had to make my own.

WPM: When did you decide to create a commercial version?Saari: That’s been around ... two years or so. So, it’s been a while. I first started asking for donations, and got some. And then it dawned on me that people might actually want to pay for this. So I came up with the premium version, and it’s been pretty good busi-ness. So it was a good move.

WPM: Why did you choose an annual subscription model for Relevanssi?Saari: I was first going to submit Relevanssi to one of those plugin marketplaces, but I got tired of waiting for them to process it, so I made it available by myself. Which was a good move, because that particular mar-ketplace folded! But I got the pricing model – where you pay a fee, and then you had this monthly support payment. It was like $ 25 for the plugin and then $ 2

per month for support. I thought that was kind of a good idea, so I ended up with the annual subscription.

But I’ve since regretted that, because it makes the maintenance a lot more complicated, when you have

“I’m not really interested in having a very large userbase”Mikko Saari is the devel-oper of Relevanssi [3], an enhanced search plugin which has been downloaded from the WordPress repository over 300,000 times.



to wait about people’s subscriptions expiring, and es-pecially since my membership plugin that I use doesn’t really work with PayPal and doesn’t quite understand the subscription, so there’s lots of manual work. But I’m now kind of stuck with it, so ...

WPM: You’ve had over 300,000 downloads of Rel-evanssi – how quickly did it pick up, and what do you think makes it so popular?Saari: Well, it developed slowly I suppose, because ... It took four years to reach 300,000 downloads, and one thing that of course helps [to increase that number] is releasing lots of versions, so people have to download it again and again and again! [laughs] That’s always a good strategy. I dunno, it’s slow and steady. One thing that has worked in my favour, is that I do a lot of ver-sions and so the plugin looks like it’s being maintained actively, and I do a lot of work for free on the Word-Press.org support forum. So, that looks good. And the competition isn’t strong – there aren’t that many good search plugins available.

In retrospect, I might have picked a better name for the plugin! Something that instantly says ‘this is about WordPress search’. Then again, I’m kind of stubborn that way. There’s so much English in the Finnish life these days, so I thought I’d fight back a bit.

WPM: What percentage of free users upgrade?Saari: It’s a very small part, of course ... I’ve sold slightly over 800 licenses. So at the moment the plugin is selling at about forty, forty-five licenses per month. So that’s fine. Of course I have limits on the time I have [available] to support people, so I’m not really interested in having a very large userbase. That’s part of the reason why I discontinued the budget license, because that license didn’t include support – but it turned out that those people are the people with the most need for support, and ask the most questions.

They are the least experienced at WordPress and are the most difficult users to help. So I quit that [pricing tier] and I’ve been very happy since! [laughs]

WPM: If you wanted to, do you think you could turn Relevanssi into a full-time job?Saari: Yeah ... it’s kind of close at the moment. I could just about, almost, feed my family with just Relevanssi. And that’s pretty good, because if I worked on it more and put more effort into the marketing part of the equa-tion, then I could easily see it would make enough money to be my prime source of income. And it’s kind of been my backup plan – if my current job fails, then I can continue [to] put more effort into Relevanssi.

WPM: Would you consider selling it off?Saari: I dunno. I have spoken to a WordPress plugin de-veloper who was kind of looking for me to buy his plugin, or vice-versa, but I didn’t go for that at the moment, and I’m not sure if I’d want to. But if I got a good enough deal, then yes. I’m not sure! This is a fun little hobby for me, so I’m quite happy con-tinuing as it is now.

Web & PHP Magazine: How long have you been working with WordPress?Michael De Wildt: Basically, two years. I had a blog for many years before that, but I didn’t decide to put a plugin into the directory until pretty much two years ago, in April. So, pretty much to this weekend. That was Backup to Dropbox, my first and only one.

WPM: Why did you start developing Backup to Dropbox? De Wildt: It was a personal need. So basically my blog got hacked by some script kiddies, and got defaced, and at that time I didn’t have a backup solution, and so I thought I’d better go get one. And basically, in the plugin directory, all the ones that worked were either insecure – for example asking for your Dropbox pass-word in plain text and storing it in wp-options [table]. Some of them just blatantly didn’t work, and the other ones were just way too complicated like that. Plugin developers seem to have a thing where they really just want to add every single feature that everybody asks for, and it’s just overcomplicated and makes things re-ally hard for the user. So I basically cracked the shits and wrote my own, that I wanted to be super-simple and just do one thing, and one thing only.

WPM: When did you decide to commercialise it?De Wildt: Well, I wouldn’t say I’ve commercialised it. Basically, after about six or seven months, I de-cided that the support load and the costs of support-ing it had started to go up and I had to start making some money off this to continue to motivate myself to solve the problems for everyone, as well as cover some of my costs.

That’s when I started thinking about ways to turn it into money, and I came up with the extensions model as a way of doing it. I kind of disagree with how the cur-rent marketplace does premium extensions. Basically

“The biggest challenge is the support load”Michael De Wildt is the developer of WordPress Backup to Dropbox [4], which has has been download over 300,000 times in the past two years.



what they do is they pull it out, their free version, they pretty much don’t support any more, and then make a premium version and try and get people to download that instead. I didn’t really want two codebases, so I decided just to continue with the 100 % free version, which is the core, which is [what] 99.9 % – no, 99 % – of my users use. And then there’s about 1 % that have actually have opted to purchase extensions.

WPM: What challenges have you faced while de-veloping the plugin?De Wildt: So basically, the biggest challenge is the support load. Once you get up to 400,000 down-loads, 80,000 users, for one person that’s a significant amount of support [to provide]. I get probably about two or three emails a day, two or three posts a day, all this sort of stuff that I have to try – not to mention development as well! So it kind of comes to a balanc-ing act of: how much time do I spend the issues that people are whinging about, and how much time do I spend telling people that I’m fixing the issues they’re whinging about? If I ignore them, they crack the shits and leave, but if I respond to them then I don’t have time to fix the problem they’re whinging about! So it’s kind of like a catch-22 there.

WPM: What percentage of free users purchase pre-mium extensions?De Wildt: Around about 1 % of my 80,000-odd users have purchased an extension. So the cool thing about that is that that small 1 % gives me enough money to cover my costs and enough motivation to still contrib-ute to the free version. So when people ask “is your plugin free?”, I usually say, “yes, it is”. Because the core version is feature-complete, and it’s free, and it always will be.

The paywall acts in two successful ways: One, obvi-ously it covers my costs, and motivates me with a little

bit of money on the side. It also stops people from us-ing zip unless they know that they’re – it makes them do their due diligence.

WPM: Is Backup to Dropbox still very much a side project, or are you considering taking it further?De Wildt: It’s becoming more than that. Like, it’s definitely got momentum and it’s definitely got merit. So, I’m actually in the tough decision at the moment to – whether or not to walk away from a consistent stable income into the unknown of a startup around a plugin. Y’know, it’s interesting what you say about BraveNewCode – these guys can run a five-person company, that’s pretty cool.

So it has changed my life in a way that it’s started to put the question on the table: should I take this further, and ... I think it’s going to take a bit more thought, and a bit more due diligence, but I think that the answer is kind of there. You never know, in six months’ time, I’ll have a real answer for you!

References

[1] http://www.bravenewcode.com/wptouch/

[2] http://wordpress.org/extend/plugins/wp-to-twitter/

[3] http://wordpress.org/extend/plugins/relevanssi/

[4] http://wpb2d.com/

PublisherSoftware & Support Media Ltd

Editorial Office Address86 Great Suffolk StLondon SE1 0BEwww.sandsmedia.com

Editor: Elliot Bentley ([email protected])Authors: Sebastian Bergmann, Frédéric Harper, Cory Isaacson, Julien Pauli, Derick Rethans Creative Director: Jens MainzLayout: Tobias Dorn, Petra Rüth

Sales:Ellen May+44 207 199 [email protected]

Contents copyright © 2013 Software & Support Media Ltd. All rights reserved. No part of this publication may be repro-duced, redistributed, posted online or reused by any means in any form, including print, electronic, photocopy, internal network, Web or any other method, without prior written permission of Software & Support Media Ltd.

The views expressed are solely those of the authors and do not reflect the views or position of their firm, any of their clients, or Publisher. Regarding the information, Publisher dis-claims all warranties as to the accuracy, completeness, or ade-quacy of any information, and is not responsible for any errors, omissions, in adequacies, misuse, or the consequences of us-ing any information provided by Pub lisher. Rights of disposal of rewarded articles belong to Publisher. All mentioned trademarks and service marks are copyrighted by their respective owners.

About Web & PHP Magazine


Documents

Preparing for the unthinkable Introduction to MongoDB · Introduction to MongoDB Querying the case for document databases PHP From the Inside Out Learn how PHP’s guts fit together