For all of the APIs I write, I use the awesome FRAPI API framework; and have been hacking away adding new features and fixing bugs more and more frequently over the last few months.

One such feature, was the addition of mimetype support. Mimetype support allows you to specify mimetypes allowed in the Accept header, and to which format the response would be.

The reason behind this was that at one point I was working with Githubs API which uses mimetypes extensively for various reasons; and at the time, I thought they were very good reasons.

A full github mimetype looks like: application/vnd.github[.version].<param>[+json]

So lets break this down:

application/

The media type (this indicates it is intended for an application to work with the data)

vnd.github

As specified in RFC 2048, vendor specific mimetypes should be prefixed with vnd. followed by the producers name

[.version]

Next up, we have [optionally] the API version, currently 3

.<param>

This is followed by a value that designates the response data type (full, raw, text, or html)

[+json]

We finish with the [optional] serialization, always json.

In theory, this looks great; just parse the mimetype and you have the version number, the response type, and potentially any number of serializations (think: XML, jsonp, serialize PHP even!). Simple, right?

Well… not quite. Lets look at an example API request (using the awesome httpie):

The important line to note here, is Content-Type: application/json; charset=utf-8.

Now lets try again, this time, with the Accept: application/vnd.github.3+json. What this says is: I want you to serve me content of this type.

Notice that we get the same type of response as before, application/json… but this isn’t what we requested. While it’s what we expect (because we read the docs and we know the entire API uses json); why isn’t the response Content-Type: application/vnd.github.3+json?

Most likely, it’s because most clients that understand json (for example those that unserialize it in to native data structures automatically) look for two types: text/json and application/json, others like jquery just look for anywhere in the Content-Type response header.

What’s wrong with this? Well, for starters, what if you request Accept: text/plain and you get back Content-Type: image/jpeg. That’s not going to work. Secondly, for caching proxies and such, it is isn’t possible to differentiate between the application/vnd.github.3.full response, and the application/vnd.github.3.raw or application/vnd.github.4.full response.

It would be entirely reasonable to respond with a HTTP/1.1 406 Not Acceptable header in the case where the server cannot respond with an accepted media type.

All in all, I think this use of mimetypes is less than great. Not terrible; definitely within the confines of the spec. So what is the solution?

The solution is part of the MIME RFCs (there are 5 original RFCs, 2045-2049, and dozens of updates) and explained in this context, in the HTTP 1.1 spec (RFC 2616), and that solution is accept-params or mimetype parameters. You’ve seen, and used them, regularly I’d bet, here’s an example:

Content-Type: text/html;charset=utf-8

These exist explicitly to pass variable named values along with the mimetype. The one explicitly defined in the HTTP spec is the q parameter for designating preference for mimetypes in the Accept header.

So lets try the github mimetypes using parameters instead:

application/json;version=3;response=raw

And in the response:

Content-Type: application/json;version=3;response=raw

Not only do we get back the same mimetype, any decent HTTP client will still see the application/json, and caches will respect the parameters, just like they would the charset parameter. You could even drop parameters to indicate it was ignored; add parameters to give more details (e.g. charset) or change the values to indicate what you actually did.

Better yet, we can now do things like use q-values. So, should Github add XML output to their API, we could indicate we can handle both, but prefer json:

Accept: application/xml;q=0.8;version=3;response=raw, application/json;q=1.0;version=3;response=raw

Or that we prefer (the future) version 4, but can still handle version 3:

Accept: application/json;q=1.0;version=4;response=raw, application/json;q=0.5;version=3;response=raw

All-in-all, I believe this to be more semantically and technically correct, while also being easier (parameters are named (case-insensitive), optionally quoted, and can be in any order except q which must be first)

I am adding another date to my summer conference circuit; this time the two-day phpDay conference in Verona, Italy on May 18th and 19th.

This time I will be presenting on PHP Streams; this is will be an in-depth look in to the streams layer and is a talk I last gave back in 2008, so there is lots of new material to cover!

I love PHP streams, definitely one of the most underrated part of PHP in my experience.

Again, if you are able to make it, do come and say Hello!

Also, I will also be at php|tek in Chicago between May 22nd-25th, however I am not speaking. (Actually, I will be there from the 20-21st to the 26th, if you want to hang out!)

I will be presenting the same talks at both conferences, “PHP 5.4: The New Bits”, and “Fast, Not Furious: Performance Tuning That Works”; so if you can’t make it across the atlantic one way or the other, you can still catch me.

If you are able to make it, find me and say hello. If you’re not able to make it, sacrifice a goat or some such and change things so you can — both of these conferences are “must-goes” if you want to learn, have fun, and engage with the awesome PHP community.

ALso, if you attended the talk, any feedback on Joind.in would be great, for friday and saturday.

However, what wasn’t made clear, is that this change was actually a backwards compatibility break. If you upgraded to 5.3.7+ data hashed pre-5.3.7 would no longer match data hashed post-5.3.7; this means if you use it for passwords, it will no longer match.

So what’s the deal here?

To encrypt using blowfish in pre-5.3.7, you would prefix your salt with $2a$ followed by a 2 digit cost parameter (base-2 logarithm, between 04-31), another $ sign, and finally 22 alphanumeric characters (0-9, A-Z, a-z).

So you end up with something like this: $2a$10$22randomcharactershere$

With PHP 5.3.7, the behavior of crypt() when using the $2a$ has changed to promote the new, more secure behavior — this is where the backwards compatibility break comes in. Now $2a$ will use the correct behavior, and has explicit countermeasures against the insecurities of the old behavior.

So what do you do when you’ve got these already hashed passwords? Reset all your user passwords? That suck!

Luckily, two new prefixes were introduced, $2x$, which exactly replicates the behavior of $2a$ in pre-5.3.7 and $2y$ which is secure like the new $2a$ but with no countermeasures.

What this means is, you can simply do the following to upgrade your passwords on-the-fly:

Now, this code assumes that you spotted the issue before updating the server; if you have had new passwords created since the upgrade, you will want to check against the $2a$ hash if the $2x$ check fails, and then update to $2y$.

The reason I chose to use $2y$ is that you can identify non-upgraded passwords by the old $2a$ prefix. I would recommend that after say, 3 months, you might want to examine your dataset for those still using $2a$ and then think about forcing password changes.

While re-using the same password might not be as secure as we would like, this ensures minimal user impact, and an easy upgrade path.

For better security, rather than automatically upgrading the hashes, you could check for $2a$ and require a new password be supplied before continuing.

The biggest change that people have been asking for, is access to $this when the closure is defined inside of an object method. This has been added:

Simple enough. However, it didn’t stop there; there was also the addition of Closure::bind() and Closure->bindTo(). These methods are identical except one is a static method into which the closure is passed, the second an instance method on the closure itself.

These methods both take two arguments (on top of the closure for the static version): $newthis and $newscope.

What this means is that unlike the regular object model the concept of $this and lexical scope (what is in scope for the function with regards to private/protected methods inside objects) are completely separated.

You can change $this to be a different object, while leaving the scope as the current object which if you then pass the current object in gives the different object access to the private/protected methods of the current method. That sounds complicated right? It is. A more likely scenario is to keep $this the same while changing the scope to another object (or static) such that you can pretend you’re accessing it from that other object and not hit private/protected methods.

Either of these scenarios is great for unit testing — I’m sure we’ll see this creep into PHPUnit sooner rather than later (I’m pretty sure Sebastian was promoting access to private/protected methods in reflection at some point — this makes that moot).

However, this can be seriously confusing, consider the following code that intentionally busts the OO model we know and love:

We have two classes, Bar with two methods, public function A() { } and private B() { }, we then have Bat which uses the __call() magic method to act as a proxy for Bar, as well as public function C() and private function D().

Inside __call() we instantiate an instance of Bar as $bar and then if __call() is able to call the requested function on $bar it simply does so, otherwise it creates a closure, passing in both the requested function name and $bar. Then using Closure::bind() we leave $this as the current object, but change the lexical scope to be that of $bar. The result of this is a new closure that is assigned to $bustOO

What this means is that inside the context of the new closure, any call on $this is referencing the object in which it was originally instantiated, but any calls on the passed in $bar will be the same as calling $this from inside $bar itself.

This means that the call to the private $bar->B() will now work. Also, the call to $this->C() works (calling Bat->C()), but things get tricky when we call $this->D().

When this happens, because the lexical scope is $bar, it cannot access Bat->D(), which means it calls Bat->__call(). Inside __call() the is_callable(array($bar, $function)) check fails (the method doesn’t exist on then tries to call $bar->D() through the callback, resulting in Fatal error: Call to undefined method Bar::D().

The full output looks like this:

Talk about head spinning!

This is the first year they are doing a multi-day conference and I’m excited to be giving my talk on PHP 5.4: The New Bits.

You can read more details on my talk and all the other great talks on their website.

I’m looking forward to my first major experience with my home countries community, please come and say Hello!

I met a bunch of really great people, and had a blast! If you’re in the area, you should check it out.

As promised, here are my slides.

You use SplFixedArray like so:

The SplFixedArray class provides a super-fast, fixed size array implementation. There are some limitations however, first you must use numeric keys and secondly you cannot use anonymous assignment (i.e. $array[] = 'value';).

You’ll notice one requirement was missing, that it should have a fixed size. While having a fixed size is what will bring you the speed increase it’s actually not a requirement that the size be fixed. Though you must specify a size to the constructor, you can change it (and lose most — if not all — speed benefits) at any time using SplFixedArray->setSize().

So, what sort of speed increase are we talking about? In my testing of arrays 100, 100… 1,000,000 elements, you will see a speed increase about 20-25%; for arrays smaller than 100, it will actually be slower by 25-40%.

The benchmarking was very simple, a comparison of a read and write iteration for both normal and fixed arrays of different sizes like so:

Additionally, the memory usage to run the benchmarks for array vs SplFixedArray is significantly different, regular arrays clock in at 198MB while SplFixedArray uses a mere 83MB, that’s a 59% memory saving.

In practical terms, you’re only going to be worried about the speed of arrays when you’re dealing with larger arrays anyway, so the speed loss for the lower digits isn’t a big concern… but where exactly could this be useful?

There is one common scenario where you will commonly be dealing with large numerically indexed arrays of data: Your database result sets. Using PDO, you can tell how many results you have before you retrieve the row data using PDOStatement->rowCount().

Unfortunately, it is not possible to set the result set container for PDOStatement->fetchAll() to use SplFixedArray — however, if someone wants to help (that is, someone who knows internals and… well, C), I’ve got an opening for a coach!

At the urging of my co-worker Helgi, I threw the arrays into a FilterIterator and got some pretty interesting results. Using similar code to the first benchmark, but instead of just reading out the array, we created and used a custom FilterIterator:

For regular arrays, we must first create an iterator:

For the SplFixedArray, we passed it straight into the EvenFilterIterator, otherwise the code is the same.

Even with the extra overhead of creating the ArrayIterator, the SplFixedArray is only marginally (1%) faster till it reaches the 10000 elements mark, and then it starts to become marginally slower (again 1%). So, I guess the take-away is: use with caution.

I’m really excited about working with many talented people, on a really cool project.

However, the best part is that I get to join a company who not only “gets” the community, but embraces and participates in it. The community and their presence in it is core to the working philosophy of the company… it was obvious in every interaction I’ve had to date and I’m really excited to be contributing to it in the future.

On that note, look for me around ZendCon in a few weeks and come and say hello!