&yet

The NoSQL "movement" in database design has been motivated by many factors (such as simplicity and scalability) and has resulted in many more choices among storage and retrieval solutions. Instead of a one-size-fits-all approach, you can choose a database that is optimized for your specific needs.

So what are your needs, and how do you choose the right database for the job?

If you don't need a database cluster, and can live with a single node and snapshot backups, then you can pretty much do whatever you want.

The CAP Theorem

As you probably know, we run Talky, a free videochat service powered by WebRTC. Since WebRTC is still evolving quickly, we add new features to Talky roughly every two weeks. So far, this has required manual testing in Chrome, Opera, and Firefox each time to verify that the deployed changes are working. Since the goal of any deploy is to avoid breaking the system, each time we make a change we run it through a post-commit set of unit tests, as well as an integration test using a browser test-runner script as outlined in this post.

All that manual testing is pretty old-fashioned, though. Since WebRTC is supposed to be for the web, we decided it was time to apply modern web testing methods to the problem.

The trigger was reading two blog posts published recently by Patrik Höglund of the Google WebRTC team, describing how they do automated interop testing between Chrome and Firefox. This motivated me to spend some time on the post-deploy process of testing we do for Talky. The result is now available on github.

Let's review how Talky works and what we need to test. Basically we need to verify that two browsers can connect to our signaling service and establish a direct connection. The test consists of three simple steps:

Next Tuesday, &yet is hosting a blood drive put on by the American Red Cross.

For some, that's maybe the most boring sentence they've ever read.

For others, who have been on a different side of that need, it's another chance.

Chance? Need? Blood?

More and more application developers have come to rely on platform-as-a-service providers for building and scaling software.

WebRTC's complexity makes it ripe for this kind of approach, so it's no surprise that so many early WebRTC companies have been platform service providers. Unfortunately for customers, the nascent Rent-Your-WebRTC-Solution market has proven pretty unstable.

News came yesterday that yet another provider of WebRTC hosted services—in this case, Requestec—has been acquired. We've seen this movie before with Snapchat's acquisition of AddLive and we'll probably see it again, maybe multiple times.

At &yet, we've been working steadily at creating open source software and approaches to infrastructure to help our clients avoid the volatile WebRTC rental market.

At &yet, we've always specialized in realtime web apps. We've implemented them in a wide variety of ways and we've consulted with numerous customers to help them understand the challenges of building such apps. A key difference is that realtime apps need a way of updating the application without direct intervention from the user.

Growing Pains

What data you send, and how much you send, is completely contextual to the application itself. Your choice of transport (polling, long-polling, WebSockets, server-sent events, etc.) is inconsequential as far as updating the page is concerned. App experience and performance are all about the data.

In our earliest experiments, we tightly coupled client logic with the updates, allowing the server side to orchestrate the application entirely. This seems rather "cool," but it ends up being a pain due to lack of separation of concerns. Having a tightly-coupled relationship between client and server means a lot of back and forth, nearly infinite amounts of pain (especially with flaky connections), and too much application orchestration logic.

On the &yet Ops Team, we use Docker for various purposes on some of the servers we run. We also make extensive use of iptables so that we have consistent firewall rules to protect those servers against attacks.

Unfortunately, we recently ran into an issue that prevented us from building Dockerfiles from behind an iptables firewall.

Here's a bit more information about the problem, and how we solved it.

The Problem

One of the best tools to use every day for locking down your servers is iptables. (You do lock down your servers, right? ;-)

Not using iptables is akin to having fancy locks with a plywood door - sure it is secure but you just cannot know that someone won’t be able to break through.

To this end I use a small set of bash scripts that ensure I always have a baseline iptables configuration and items can be added or removed quickly.

Let me outline what they are before we get to the fiddly bits...

Building collaboration apps has always been too hard, especially when audio and video are involved. The promise of WebRTC - building audio and video support directly into the browser - has been that it would make collaboration as easy and open as the web itself. Unfortunately, that hasn't quite happened yet.

A big part of the problem is that we lack the kind of open platforms and frameworks that have made the web such a success - things like, say, express and hapi in the Node.js community. Instead of open standards and open source, web developers coming to the collaboration space find themselves faced with a plethora of proprietary platforms tied to specific vendors.

Don't get us wrong: some of those platforms are excellent. However, if you're dependent on a particular service provider for the platform that powers your entire real-time collaboration app then you've taken on some significant risks. Vendor lock-in and high switching costs are two that come quickly to mind. Not to mention the possibility that the startup service you depend on will go out of business, get bought by one of your competitors, or be taken off the market by an acquiring company.

That's not a recipe for happy application developers.

About a year ago, our friends at TokBox published a blog post entitled WebRTC and Signaling: What Two Years Has Taught Us. It's a great overview of their experience with technologies like SIP and XMPP for stitching together WebRTC endpoints. And we couldn't agree more with their observation that trying to settle on one signaling method for WebRTC would have been even more contentious than the endless video codec debates. However, our experience with XMPP differs enough from theirs that we thought it would be good to set our thoughts down in writing.

First, we're always a bit suspicious when someone says they abandoned XMPP because it didn't scale for them. As a general rule, protocols don't scale - implementations do. We know of several XMPP server implementations that can route, distribute, and deliver tens of thousands of messages per second without breaking a sweat. After all, that's what a messaging server is designed to do. Sure, some server implementations don't scale up that high; the answer is to use a better server. We're partial to Prosody, but Tigase and a few others are well worth researching, too.

Second, pure messaging is the easy part. The hard part is defining the payloads for those messages to build out a wide variety of features and functionality. What we like best about the XMPP community is that over the last 15 years they have defined just about everything you need for a modern collaboration system: presence, contact lists, 1:1 messaging, group messaging, pubsub notifications, service discovery, device capabilities, strong authentication and encryption, audio/video session negotiation, file transfer, you name it. Why put a lot of work into recreating those wheels when they're ready-made for you?

An added benefit of having so many building blocks is that it's straightforward to put them all together in productive ways. For example, XMPP includes extensions for both multi-user chat ("MUC") and multimedia session management (Jingle). If we need multiparty signaling for video conferencing, we can easily combine the two to create what we need. Plus we get a number of advanced solutions for free this way, since MUC includes in-room presence along with a helpful authorization model and Jingle supports helpful features like renegotiation and file transfer. Not to mention that the ability to communicate device capabilities in XMPP enables us to avoid monstrosities like SDP bundling.

As a portion of our elaborate training events I give a short talk about JS frameworks. I've shied away from posting many of my opinions about frameworks online because it tends to stir the pot, hurt people's feelings, and unlike talking face to face, there's no really great, bi-directional channel for rebuttals.

But, I've been told that it was very useful and helped provide a nice, quick overview of some of the most popular JS tools and frameworks for building single page apps. So, I decided to flesh it out and publish it as A Thing™ but please remember that you're just reading opinions, I'm not telling you what to do and you should do what works for you and your team. Feel free to disagree with me on twitter or even better, write a post explaining your position.

Angular.js

pros