I’ve now spent the better part of four months looking at HTTP/2 (or H2 as all the cool kids are calling it) and in particular Server Push.
Server Push is super exciting to me, paired with multiplexing — the ability to perform multiple HTTP transactions on a single TCP connection — I believe it has the potential to change how we deliver the web by allowing the server to push content to the client (specifically, into the client cache) proactively.
How Server Push Works
To explain Server Push, you must first understand HTTP/2.
HTTP/2 requests are sent via streams — a stream is the unique channel through which a request/response happens within the TCP connection. Each stream has a weight (which determines what proportion of resources are used to handle them) and each can be dependent on another stream.
Streams are made up of frames. Each frame has a type, length, any flags, and a stream identifier, to explicitly associate it with a stream.
The ability to interleave these frames, and the fact they can belong to any stream, is the basis of multiplexing.
One of those frame types is the SETTINGS frame, which is how the client can control whether or not to use Server Push.
Server Push can be enabled (default), or disabled by sending the SETTINGS_ENABLE_PUSH (or 0x02) flag.
When it is disabled, the server should not send any pushes. When it is enabled, a push is started by sending another type of frame, known as a PUSH_PROMISE.
The purpose of this frame is to inform the client that the server wants to push a resource, and to give the client the option to reject the pushed stream (by sending back a RST_STREAM frame). Each pushed resource is then sent in it’s own stream to the client and should be stored in the client cache — it will then be retrieved from the cache when it is requested by the client rather than fetched from the server.
Pushed resources must be cacheable as it is required to still be fresh when the actual request occurs. This means they should be the result of idempotentGET requests.
That’s Cool and All, but it’s Not Revolutionary…
As I said, I think Server Push and Multiplexing can change the web.
In the near term, we can start to simplify our web setups; multiplexing obsoletes domain sharding (in fact, sharding can be a detrimental practice, though not always), as well as a number of frontend strategies for performance tuning, such as inlining of resources for above-the-fold rendering, image sprites, and CSS/JS concatenation and minification.
Thinking longer term, we will start to see new strategies emerge, such as pushing the above-the-fold JS/CSS as separate resources with high priority along with the requested HTML, followed by the rest of the CSS/JS with a lower priority.
Or making webfonts dependant on the CSS file in which they are being used.
But The Web Isn’t Just Websites…
Another casualty of HTTP/1.1 is APIs. APIs often have to make the choice between combining sub-resources into the parent resource (sending more data than necessary if they are not be wanted, slowing down the response), or making many more requests for those sub-resources.
With Server Push and Multiplexing, the server can push those sub-resources along with the request for the parent resource, and the client can then choose to reject them if it doesn’t want them.
Alright, but what do you mean we’re doing it wrong?
Currently, the most popular way to do server push is for your application to send Link: /resource; rel=preload headers, which will inform the http server to push the resource in question. However, this format is defined by the new W3C Preload Specification (Working Draft), which is not intended for this purpose (although there is some disagreement).
The purpose of the preload Link is for a browser and:
provides a declarative fetch primitive that initiates an early fetch and separates fetching from resource execution.
It is related to the (so-called) Prebrowsing features — which allow you to instruct the browser to do a number of things to improve the performance of sub-resources (everything from pre-emptively doing a DNS lookup, opening a TCP socket, to fetching and completely pre-rendering the page in the background).
I like the solution of using headers to initiate pushes. This makes it something that can easily be done in non-async/parallel/threaded languages (e.g. PHP or Ruby) — with zero language changes necessary — and pushes the responsibility up to the HTTP layer.
Unfortunately, you run into a potential issue of being unable to distinguish between preloading and server push; and you may wish to use both for different assets — for example, you might want to use prefetching for your stylesheet, which when retrieved could have it’s fonts and images pushed. Furthermore, using preload for pushes could introduce a race condition between the a push and a preload for the same resource.
We don’t want to clobber the Preload specification, so: why not just change it to Link: /resource; rel=push.
By doing this we add enough granularity to distinguish between the two, and avoid a potential race condition. The header would be stripped by the httpd when it handles the push. If the client does not accept pushes (which the server knows thanks to the SETTINGS frame) the header should be passed through as-is (or can be changed to rel=preload) and the browser can then handle it as a preload instead.
If neither preload or push is supported then the asset is requested using the traditional methods (e.g. link, script, and img tags, or url() within stylesheets) this allows for a simple, robust, progressive enhancement mechanism for the loading of assets.
This was going to just be a tweet, but I felt it needed more than 140 characters.
If you know me in person, you’ve probably met me at a conference or user group. It should be immediately apparent that I am a confident, outspoken person (at least publicly).
I’m a former PHPWomen US Lead, as well as co-organizer for Prompt, raising awareness about mental health in technology, I’m no stranger to speaking out on tough subjects.
Furthermore, I’ve been a contributor to the PHP community for 15 years, and an internals contributor for half of that (since PHP 5.3.0).
The fact that I — as that person, and as a member of the majority makeup of the PHP community — don’t feel safe contributing back to the Code of Conduct discussion in favor of it — should tell you everything you need to know about why we need one. This is due in part to the hostility from some people involved, and to possible backlash from the wider community.
So let me state here, for the record, in unequivocal terms:
I fully support a Code of Conduct, and a reporting infrastructure in which we can act to remove those who are unable to abide by the Code of Conduct from our community.
This should be applied to all mailing lists, all websites (e.g. comments in the manual, news items, etc.), and all social media put out by the PHP project itself.
It should also apply anywhere a contributor is using their @php.net email address — whether thats a private email conversation, or at a conference/user group where you put it on your contact slide. Obviously while the person could not be removed from the event, they can be restricted from contributing based on their behavior.
Also: thank you Anthony for bringing this RFC to the table.
This has gotten quite a bit of support and a couple of questions, so I wanted to add one specific thing:
To clarify, I don’t support the Contributor Covenant as it stands as the CoC for PHP; I think it is a good starting point, but it is just that, a starting point. Additionally, regardless of the content of the CoC, what matters more is documented procedures for handling reported violations.
As 2015 comes to a close, I thought that I would jot down some words to reflect up what can only be called an incredible year.
In April I will celebrate the end of my third year as a Community Engineer Developer Evangelist, and it has been filled with more firsts that I ever thought I would experience.
The year started out pretty standard with PHPBenelux (not that one could ever even suggest that PHPBenelux is anything but an extraordinary event!), but was then followed by the first of threeWurstCons, which was actually a mini-tour of Germany.
We took the train from Brussels to Köln, where we spent two nights and visited a local Biergärten, an experience in and of itself. Here, I tried blood sausage for the first time (which some might find strange, as I am British), as well as pork knuckle.
From Köln, Yitz, Beth, and myself took another train to Frankfurt, while Jeremy flew to Berlin. We spent one night in Frankfurt, before Beth went off, leaving Yitz and I to carry on and meet Jeremy in Berlin.
After 3 days in Berlin, and 6 days total in Germany, I took the train back to Brussels for FOSDEM.
Next up, was ConFoo, where I unfortunately had to rush through (one talk before lunch, one talk after, and then a flight 2 hours later…) on my way to PHPUK where I was giving my first keynote.
I cannot thank the PHPUK folks enough for allowing me to have such an amazing experience. This was a professional goal of mine for many years, and I finally realized it on a stage that, being in my home country, made it all the more special. Plus, my mum got to come!
March started with me knocking another two items off my bucket-list: I made it to the Southern Hemisphere, and specifically, to Australia, for PHPAustralia. In Sydney, I got to put my feet in the other side of the Pacific Ocean for the first time, as well as see Koalas, Tasmanian Devils, Wallabies, and even take a Quokka selfie with Jordi Boggiano at Taronga Zoo.
I saw the Sydney Opera House and the Sydney Harbor Bridge, another bucket list item (the Opera House is much smaller than I thought!), and the Three Sisters.
I even ate Kangaroo (both in Kebab, and steak form, very yummy!), as well as Crocodile (tasted like salt water…).
Following on from my longest trip to date, I then took one of the shortest, in what was my least busy month of the year, to one of my favorite conferences, Lone Star PHP. Once again, Lone Star delivered, and I have the nice comfy t-shirt to prove it. I also experienced my first Tornado Siren. That was not on my bucket list.
We also enjoyed one of the best outings for PHP Karaoke for the year. I think this picture from Ben Ramsey captures it pretty well:
I also saw this great talk by Heather White on Teaching, which changed the way I do my slides and I’ve been getting compliments ever since: Thanks Heather!
On the way home from Lone Star, I actually ditched my checked luggage in Atlanta and went to hang out at my first RailsConf, and in fact, my first Ruby-related conference. I had a great time, met some great people, and got to hang out with PJ again. Oh yeah, and Phil Sturgeon turned up too. More karaoke happened.
May was another month for knocking out bucket-list items, I took my wife along to Rome, Italy where we got to see the Vatican City and watch the Pope speak (neither on my bucket list, but highly recommended), I also got to see The Colosseum (the oldest item on my bucket list, I actually cried), Palatine Hill, Trevi Fountain, and much more. From here, we took a train to Venice, another bucket list item, and got to see the islands of Murano (famous for its glassware) & Burano (famous for its brightly colored painted buildings), as well as Piaza San Marco, and some beautiful sunsets and canals views.
Our final stop in Italy, was of course Verona, for phpDay, a great little conference, in a lovely town with it’s own historic arena (which is apparently older than The Colosseum), as well as Juliets House, and I got to see the old city for the first time at the speaker dinner.
From there, I headed on to Chicago, for php[tek], always a good time, and in fact, the last time it will be held in Chicago. I am excited to see what St. Louis brings to the table, and how we will ever replace Shoeless Joe’s. I think I will always remember watching The EuroVision Song Contest with a small group of people on my phone in the Club Lounge after the conference ended…
After a quick trip to Boston, MA, I headed down to Mexico for MagmaConf, an amazing polyglot conference that is only eclipsed by it’s location. If you ever get the chance to go, take it. I had an amazing time for my first time in Mexico (other than the port of Cozumel on a cruise, which I’m not counting) — I flew into Colima, and then, after giving a workshop, took a bus through the hills to Manzanillo on the Pacific coast. MagmaConf happens at Magma Village, a coastal resort where you share a villa with a number of other [random] folks attending the conference. I swam in 7 different pools over the 2 days of the conference, hanging out and meeting new people.
I then ended the month at the DutchPHP Conference, this was my second time there, and it turned out to be my last talk for Engine Yard. This was actually quite an emotional thing for me — after almost 4 years, I was no longer a Yardee.
But, thanks to that quick trip to Boston back in May (Cambridge, actually), I was moving on to a new gig at Akamai Technologies. I then experienced yet another first, this time, Istanbul, Turkey. This was my first experience of a country that didn’t use the Latin alphabet (except Egypt, before I could read 😋), which was quite an interesting experience, and it would not be my last time this year.
Istanbul was another amazing city, with so much history and culture. I got to see the underground Basilica Cistern, and sunset from Galata Tower, among other great sights, and attended another great conference, PHPKonf. This was also the first time speaking where I was translated in real time, which was fascinating.
August was the start of my longest travel period ever. Starting with a family vacation on the 24th, I was on the road non-stop till October 9th.
Considering I started the year never having been to the Southern Hemisphere, I — much to my amazement – got to go a second time. This time, I headed to beautiful New Zealand, and while I didn’t get to see Hobbits, I did get to enjoy a lovely drive along the coast, and had a fantastic time catching up with friends new and old at the New Zealand PHP Conference.
Contrary to popular belief, I did not write the patch for T_SPACESHIP in 7.0. I wrote and proposed the T_SPACESHIP against 5.6, and it was re-written independently by Andrea Faulds. She later took on the T_SPACESHIP name after finding my original patch.
P.s. it’s one of these, not a Tie-Fighter or anything else!)
From New Zealand, I headed back to Sydney for a few days, where I got to speak at the SydPHP User Group, and gave my first talk on HTTP/2, a topic that would become a major focus for me for the rest of the year (and hopefully, beyond!). Next, I headed back to the US, to Seattle for the first time, and to meet up with my wife and son, for the inaugural Pacific North West PHP, where I was once again fortunate to keynote.
Following the event was my second WurstCon of the year, WurstCon NW, at Pike’s Place Market, where we shut down the restaurant for our impromptu private party (thanks to the gracious staff for that!)
I enjoyed this conference a lot, and apparently enjoyed Seattle so much (especially the drive through Mt. Rainier National Park) I’m planning to move there this upcoming February where I’ll get to hang out with awesome folks like Margaret Staples and Tessa Mero.
I then flew back to Europe, to Sofia, Bulgaria, my second non-Latin language country of the year. Now, let me stop for a moment and give a special shout out to this conference, easily in my top 3 for the year. The organizers of this conference were so amazing at making the speakers feel welcome, and safe in a country where all but two of us couldn’t even read the alphabet, and the event itself was just top-notch. I absolutely have this one my must-do list next year. Also, I climbed a fricken mountain. And the group hug at the end was epic. Oh, and let’s not forget, so much food meat, including my first time eating Horse thanks to the nicest guy I know, Damiano Venturing.
Thank you to the entire crew at Site Ground, but especially Dima Peteva for being my handler, and for even translating some Cyrillic receipts via IM for expenses after I got home!
From Europe, I took my third and final trip to the Southern Hemisphere for the year for PHP South Africa.
PHP South Africa actually comprised of two events, the first in Johannesburg, and the second in Cape Town. Johannesburg while fantastic, couldn’t have prepared me for our 2-day stop before heading on to Cape Town: Pilanesburg.
Here I got to take another item off my bucket list: we took not one, but two safaris. Despite the bug in my room larger than my head (my wife who was on FaceTime with me at the time I discovered it — and deafened by my high pitched scream — says I’m exaggerating), and losing one of my lenses, this was one of the most amazing experiences I’ve ever had. Seeing Elephants, Rhinos, Lions, Leopards, Hippos, Antelope, Zebras, and more in the wild, was beyond words. No photo, zoo, or wildlife park can do them justice.
And yet… even that didn’t compare to the majesty of seeing, and ascending Table Mountain in Cape Town. Although Jordi and Derick actually climbed the mountain, myself, Raphael Dohms and his wife Tiscilla took the cable car to the top, where we were treated to amazing views of the end of the world, and after being joined by the hikers, we enjoyed watching the sun set over the water1.
My final event took me back to the frozen slightly cool North, this time to Toronto for TrueNorth PHP and the final — arguably the best — WurstCon of the year, at Wvrst in downtown Toronto, where I got to split a multi-sausage meal with Margaret and Mr. WurstCon himself, Jeremy Mikola (and we were joined by a bunch of other great folks).
We started with duck, pheasant, venison, bison, and guinea fowl:
And of course, I participated in this work of art:
Unfortunately, I was supposed to also speak at RailsIsrael (how this event isn’t called Israils, I have no idea!) but had to cancel due to safety concerns — this also means I didn’t manage to achieve my goal of hitting every continent2.
I also started my first real RFC for PHP 7.1, adding — you guessed it — yet more HTTP/2 feature support to ext/curl, this time in the form of HTTP/2 Server Push support.
December is the first month this year without any travel for work, and I’m looking forward to a full month with my family, to enjoy Christmas, and reflect on the year passed.
This year, I visited 13 countries, covering 162,862 miles (262,101km) on 79 flights via 28 airports. I spent 15.5 days in the air. That’s equal to more than six and a half (6.54) times around the world, and over half way (68.2%) to the moon!
I also tried many new kinds of meat: Kangaroo, Crocodile, Horse, Kudu, Ostrich, Springbok, Guinea Fowl, Pheasant, Venison, Bison, Wild Boar, Elk, Rabbit, and Veal!
I hope to see all of you again next year, somewhere, and I look forward to meeting new people, and learning even more than I did this year!
You can see an even better time-lapse by Jordi, here↩
At least as far the UN 5 inhabited continents model goes! ↩
Prior to November 19th 2006 I was really into PHP. I tested every alpha, beta, and release candidate, I knew every new feature inside and out, and I had plans to contribute… something. I didn’t know what, but I was going to give back to the language I loved.
However, just a few weeks after the release of PHP 5.2, my first wife passed away and I pretty much dropped off the face of the Earth as far as being part of the PHP community is concerned (EartPHP?). I did my job, and very little else. I had pushed out PHAR the year before, and it went on to become part of PHP 5.3, earning me my name in the phpinfo() output — a long time goal of mine at the time — but I pretty much missed the 5.3 release. For all the drama around killing PHP 6, and the huge changes in 5.3, I just wasn’t paying much attention.
I’ve felt like I was playing catch-up to pre-November 2006 me ever since then.
Despite everything I’ve accomplished since then, I never felt like I had reached the same level I was at back then (see: Impostor Syndrome). Till now.
To help ease the transition from 5.6 to 7.0 I have created a simple package that acts as a shim between the newly removed ext/mysql and ext/mysqli.
I was a little hesistent to even publish this as I don’t want to encourage the continued use of potentially insecure code, however, I want people to upgrade to 7.0 and don’t want this to be the blocker.
It does require 5.6 (though it would be possible to lower that) — however I suspect that most people who are upgrading to 7.0 are either coming from 5.6 or have the native ext/mysql. The primary reason for support 5.6 is to be able to compare the test suite results against native ext/mysql.
It’s not yet production ready, but tests are coming along pretty well (PRs welcome!):
Current known (and unlikely to be fixed) issues are:
You must prefix all calls to mysql_* with a \ (e.g. \mysql_connect()); Not sure why I thought this, but non-internal functions do fall back to the global namespace.
Calls to is_resource() and get_resource_type() on MySQL connections and results will fail as these are now their mysqli equivalents.
-Some errors are now from ext/mysqli, and others are E_USER_WARNING instead of E_WARNING. Where possible, the original error messages are replicated.
Column lengths reported by mysql_field_len() assume latin1 charset, and will return incorrect lengths for other charsets.
This is a simple middleware that records a request’s response the first time it’s made in a test, and then replays it in response to requests in subsequent runs.
It does this by returning a Guzzle \GuzzleHttp\HandlerStack with either the \Dshafik\GuzzleHttp\VCRHandler middleware, or the GuzzleHttp\Handler\MockHandler added. The first will record the responses to JSON files, while the latter will be pre-loaded with those responses and will return them when requests are made.
It’s important to understand that the responses are returned in order regardless of whether it is the same request being made.
The purpose of this library is to just make it easier to create and update your tests for API clients.
Usage is simple, just call Dshafik\GuzzleHttp\VCRHandler::turnOn() passing in the storage location before running the test, and pass in the handler as a guzzle client option:
You can pass in the handler when instantiating the \GuzzleHttp\Client, or when making the individual requests — if you use the same instance for the individual requests it will re-use the same JSON file for storage, otherwise if you pass in unique instances (with unique storage files) it will create individual ones. I recommend passing in the handler to the constructor, but ensuring that you use a new instance (of the middleware, and the client) for each test.
Hopefully folks find this useful, do let me know if you do. If you have issues, please report them and pull requests are welcome!
I’ll be releasing a new Akamai library which uses dshafik/guzzlehttp-vcr next week (probably) so look out for that if you want to see it’s use in a real project.