Microservices — an architectural pattern we recommended in our 2012 International PHP Conference keynote — is pretty popular these days. There are many benefits to consider:
Independent development and release lifecycle for each microservice
... [More]
Ensuring clear API boundaries between systems
Ability to use technologies most applicable for each area of a system
In an ideal world, microservices are a realization of the Unix philosophy as applied to building internet services: writing programs that do one thing, and do it well; writing programs that work together.
Just use a message queue
However, when most people think about microservices, they think systems that communicate with each other using HTTP APIs. I think this is quite limited, and something that makes microservices a lot more fragile than they could be. Message queues provide a much better solution. This is mentioned in Martin Fowler’s Microservices article:
The second approach in common use is messaging over a lightweight message bus. The infrastructure chosen is typically dumb (dumb as in acts as a message router only) - simple implementations such as RabbitMQ or ZeroMQ don’t do much more than provide a reliable asynchronous fabric - the smarts still live in the end points that are producing and consuming messages; in the services.
When we were building The Grid, we went with an architecture heavily reliant on microservices communicating using message queues. This gave us several useful capabilities:
Asynchronous processing
If you have heavy back-end operations or peak load, a microservice might be swamped with work to be done. If your web server only needs to send work to a queue and not wait for result immediately, you have a lot more freedom on how to organize the work. Furthermore, you can split your service along different scalability characteristics — some systems may be network-bound, others CPU-bound etc
Autoscaling
Since the work to be performed by your microservices is kept in message queue, you can use the combination of the queue length and typical processing times to automatically determine how many instances of each service you need. Ideally you can use this to ensure that your processing times stay consistent regardless of how many users are on your system at a given time
Dead lettering
It is possible to configure RabbitMQ to place any failed operations into a dead letter queue. This gives you a full record of any failing operations, making it possible to inspect them manually, replay later, or produce new tests based on real-world failures
Rate limiting
Sometimes you’re dealing with external HTTP APIs that are rate limited. Once a microservice starts hitting rate limits, you can keep new requests in a queue until the limit lifts again
In-flight updates
A message queue can be configured so that non-acknowledged messages go back into the queue. This means that if one of your services crashes, or you deploy a new version of it, no work gets lost. When the service is back up again, it can pick up right where the previous instance left off
Better tooling
HTTP is something all backend developers are familiar with. With message queues you have to deal with some new concepts, and hence new tools are also needed.
MsgFlo
Client libraries for talking with common message queues exist for pretty much every language. However, this still means that you’ll have to handle things like message pre-fetch limits, acknowledging handled messages, and setting up queues yourself. And of course keeping track of what service talks to what can become a burden.
We developed the MsgFlo tool to solve these problems. MsgFlo provides open source client libraries for bunch of different programming languages, providing a simple model to handle message-related workflow.
To give you an overview of the dataflow between services, MsgFlo also provides a way to define the whole system as a flow-based programming graph. This means you’ll see the whole system visually, and can change connections between services directly from graphical tools like Flowhub.
If you’re using Heroku, MsgFlo can also generate the Procfile for you, meaning that the services you’ve defined in your graph get automatically registered with the platform.
GuvScale
As mentioned above, queue-based microservices make autoscaling quite easy. If you’re on Heroku, then our GuvScale can be used to automate scaling operations for all of your background dynos.
I wrote more about GuvScale in a recent blog post.
Further reading
If you’d like to explore using message queues for your services a bit more, here are couple of good articles:
Event-driven Microservices Using RabbitMQ
Microservice Communication with Queues
There are also some MsgFlo example applications available:
msgflo-example-imageresize is a web service for scaling images using autoscaled background workers
imgflo-server is a web service that uses MsgFlo to send image processing jobs to background GEGL or NoFlo microservices
[Less]
|
As I’m preparing for a NoFlo talk in Bulgaria Web Summit next week, I went through some older videos of my conference talks. Sadly a lot of the older ones are not online, but the ones I found I compiled in playlists:
My talks on YouTube
My
... [More]
talks on Vimeo
Some highlights
NoFlo, Managing Workflows with JavaScript, my first NoFlo talk, Wakanday 2011:
Decoupling Content Management with Create and PHPCR from SymfonyLive Paris 2012:
The Internet is your Application Blueprint, keynote with Lukas at the International PHP Conference 2012 where we talked about microservices:
Introduction to Flow-Based Programming and NoFlo at c-base in 2013:
If you find any I’ve missed, please let me know!
[Less]
|
As I’m preparing for a NoFlo talk in Bulgaria Web Summit next week, I went through some older videos of my conference talks. Sadly a lot of the older ones are not online, but the ones I found I compiled in playlists:
My talks on YouTube
My
... [More]
talks on Vimeo
Some highlights
NoFlo, Managing Workflows with JavaScript, my first NoFlo talk, Wakanday 2011:
Decoupling Content Management with Create and PHPCR from SymfonyLive Paris 2012:
The Internet is your Application Blueprint, keynote with Lukas at the International PHP Conference 2012 where we talked about microservices:
Introduction to Flow-Based Programming and NoFlo at c-base in 2013:
If you find any I’ve missed, please let me know!
[Less]
|
Flowhub — the product made possible by our successful NoFlo Kickstarter — has now its own company dedicated to supporting and improving the visual programming environment.
Last fall we bought the Flowhub and other flow-based programming assets from
... [More]
The Grid, and now after some paperwork we’re up and running as Flowhub UG, a company registered in Berlin, Germany.
We’re also now selling the new Pro and Supporter plans, and can also provide a dedicated on-premise version of Flowhub. Please check out our Plans for more information:
Read more in the latest Kickstarter update.
[Less]
|
Flowhub — the product made possible by our successful NoFlo Kickstarter — has now its own company dedicated to supporting and improving the visual programming environment.
Last fall we bought the Flowhub and other flow-based programming assets from
... [More]
The Grid, and now after some paperwork we’re up and running as Flowhub UG, a company registered in Berlin, Germany.
We’re also now selling the new Pro and Supporter plans, and can also provide a dedicated on-premise version of Flowhub. Please check out our Plans for more information:
Read more in the latest Kickstarter update.
[Less]
|
I’m happy to announce that GuvScale — our service for autoscaling Heroku background worker dynos — is now available in a public beta.
If you’re using RabbitMQ for distributing work to background dynos hosted by Heroku, GuvScale can monitor the
... [More]
queues for you and scale the number of workers up and down automatically.
This gives two big benefits:
Consistent processing times by scaling dynos up to meet peak load
Cost savings by reducing idle dynos. Don’t pay for computing capacity you don’t need
We originally built the guv tool back in 2015, and it has been used since by The Grid to manage their computationally intensive AI tasks. At The Grid we’ve had GuvScale make hundreds of thousands of scaling operations per month, running background dynos at more than 90% efficiency.
This has meant being able to produce sites at a consistent, predictable throughput regardless of how many users publish things at the same time, as well as not having to pay for idle cloud machines.
Getting started
There are many frameworks for splitting computational loads out of your main web process and into background dynos. If you’re working with Ruby, you’ve probably heard of Sidekiq. For Python there is Celery. And there is our MsgFlo flow-based programming framework for a more polyglot approach.
If you’re already using one of these with RabbitMQ on Heroku (for example via the CloudAMQP service), you’re ready to start autoscaling with GuvScale!
First enable the GuvScale add-on:
$ heroku addons:create guvscale
Next you’ll need to create an OAuth token so that GuvScale can perform scaling operations for your app. Do this with the Heroku CLI tool. First install the authorization add-on:
$ heroku plugins:install heroku-cli-oauth
Then create a token:
$ heroku authorizations:create --description "GuvScale"
Copy the authentication token, and paste it to the GuvScale dashboard that you can access from your app’s Resources tab in Heroku Dashboard.
Once GuvScale has an OAuth token, it is ready to do scaling for you. You’ll have to provide some scaling rules, either in the GuvScale dashboard, or via the heroku-guvscale CLI tool.
Here is an example for scaling a process that sends emails on the background:
emailsender: # Dyno role to scale
queue: "send-email" # The AMQP queue name
deadline: 180 # 3 minutes, in seconds
minimum: 0 # Minimum number of workers
maximum: 5 # Maximum number of workers
concurrency: 10 # How many messages are processed concurrently
processing: 0.300 # 300 ms, in seconds
Once GuvScale has been configured you can monitor its behavior in the dashboard.
Read more in the Heroku Dev Center GuvScale tutorial.
GuvScale is free during the public beta. Get started now!
Rescaling 3k images. From 0 servers to 25, back to 0 in <4 mins. 0 humans involved.#devops https://t.co/OSPMZtfrd9 pic.twitter.com/s7Bog4hHLZ— Jon Nordby (@jononor) June 8, 2016
[Less]
|
I’m happy to announce that GuvScale — our service for autoscaling Heroku background worker dynos — is now available in a public beta.
If you’re using RabbitMQ for distributing work to background dynos hosted by Heroku, GuvScale can monitor the
... [More]
queues for you and scale the number of workers up and down automatically.
This gives two big benefits:
Consistent processing times by scaling dynos up to meet peak load
Cost savings by reducing idle dynos. Don’t pay for computing capacity you don’t need
We originally built the guv tool back in 2015, and it has been used since by The Grid to manage their computationally intensive AI tasks. At The Grid we’ve had GuvScale make hundreds of thousands of scaling operations per month, running background dynos at more than 90% efficiency.
This has meant being able to produce sites at a consistent, predictable throughput regardless of how many users publish things at the same time, as well as not having to pay for idle cloud machines.
Getting started
There are many frameworks for splitting computational loads out of your main web process and into background dynos. If you’re working with Ruby, you’ve probably heard of Sidekiq. For Python there is Celery. And there is our MsgFlo flow-based programming framework for a more polyglot approach.
If you’re already using one of these with RabbitMQ on Heroku (for example via the CloudAMQP service), you’re ready to start autoscaling with GuvScale!
First enable the GuvScale add-on:
$ heroku addons:create guvscale
Next you’ll need to create an OAuth token so that GuvScale can perform scaling operations for your app. Do this with the Heroku CLI tool. First install the authorization add-on:
$ heroku plugins:install heroku-cli-oauth
Then create a token:
$ heroku authorizations:create --description "GuvScale"
Copy the authentication token, and paste it to the GuvScale dashboard that you can access from your app’s Resources tab in Heroku Dashboard.
Once GuvScale has an OAuth token, it is ready to do scaling for you. You’ll have to provide some scaling rules, either in the GuvScale dashboard, or via the heroku-guvscale CLI tool.
Here is an example for scaling a process that sends emails on the background:
emailsender: # Dyno role to scale
queue: "send-email" # The AMQP queue name
deadline: 180 # 3 minutes, in seconds
minimum: 0 # Minimum number of workers
maximum: 5 # Maximum number of workers
concurrency: 10 # How many messages are processed concurrently
processing: 0.300 # 300 ms, in seconds
Once GuvScale has been configured you can monitor its behavior in the dashboard.
Read more in the Heroku Dev Center GuvScale tutorial.
GuvScale is free during the public beta. Get started now!
Rescaling 3k images. From 0 servers to 25, back to 0 in <4 mins. 0 humans involved.#devops https://t.co/OSPMZtfrd9 pic.twitter.com/s7Bog4hHLZ— Jon Nordby (@jononor) June 8, 2016
[Less]
|
I’ve been attending the Bosch Connected Experience IoT hackathon this week at Station Berlin. Bosch brought a lot of different devices to the event, all connected to send telemetry to Eclipse Hono. To make them more discoverable, and enable rapid
... [More]
prototyping I decided to expose them all to Flowhub via the MsgFlo distributed FBP runtime.
The result is msgflo-hono, a tool that discovers devices from the Hono backend and exposes them as foreign participants in a MsgFlo network.
This means that when you connect Flowhub to your MsgFlo coordinator, you have all connected devices appear there, with port for each sensor they expose. And since this is MsgFlo, you can easily pipe their telemetry data to any Node.js, Python, Rust, or other program.
Hackathon project
Since this is a hackathon, there is a competition on projects make in this event. To make the Hono-to-MsgFlo connectivity, and Flowhub visual programming capabilities more demoable, I ended up hacking together a quick example project — a Bosch XDK controlled air theremin.
This comes in three parts. First of all, we have the XDK exposed as a MsgFlo participant, and connected to a NoFlo graph running on Node.js
The NoFlo graph starts a web server and forwards the telemetry data to a WebSocket client.
Then we have a forked version of Vilson’s webaudio theremin that uses the telemetry received via WebSockets to make sound.
The whole setup seems to work pretty well. The XDK is connected to WiFi here, transmits its telemetry to a Hono instance running on AWS. This data gets forwarded to the MsgFlo MQTT network, and from there via WebSocket to a browser. And all of these steps can be debugged and experimented with in a visual way.
Relevant links:
msgflo-hono tool
Hono MsgFlo demo project
Update: we won the Open Hack Challenge award for technical brilliance with this project.
[Less]
|
It has always been easy to wrap existing JavaScript code into NoFlo graphs — just write a component that exposes its functionality via some ports.
Going the other way and exposing a NoFlo graph to JS land was also possible but cumbersome. With NoFlo
... [More]
0.8.3 we now made it super easy with the new asCallback API.
Let’s say you have a graph (or just a component) with the typical IN, OUT and ERROR ports, you can wrap it like this:
// Load the NoFlo module
var noflo = require('noflo');
// Use NoFlo's asCallback helper to prepare a JS function that wraps the graph
var wrappedGraph = noflo.asCallback('myproject/MyComponent', {
// Provide the project base directory where NoFlo seeks graphs and components
baseDir: '/path/to/my/project'
});
// Call the wrapped graph. Can be done multiple times
wrappedGraph({
// Provide data to be sent to inports
in: 'foo'
}, function(err, result) {
// If component sent to its error port, then we'll have err
if (err) { throw err; }
// Do something with the results
console.log(result.out);
});
If the graph has multiple inports, you can provide each of them a value in that input object. Similarly, the results sent to each outport will be in the result object. If a port sent multiple packets, their values will be in an array.
Use cases
With asCallback, you can implement parts of your system as NoFlo graphs without having to jump in all the way. Even with a normal Node.js or client-side JS application it is easy to see places where NoFlo fits in nicely:
Making complex Express.js or Redux middleware chains manageable
Adding customizable workflows to some part of the system
Implementing Extract, Transform, Load (ETL) pipelines
Porting a system into NoFlo piece-by-piece
Exposing a pure-JS API for an NPM module built in NoFlo
It also makes it easier to test NoFlo graphs and components using standard, non-FBP-aware testing tools like Mocha.
Network lifecycle
Each invocation of a callbackified NoFlo graph creates its own Network instance. The function collects result packet until network finishes, and then calls its callback.
Since these networks are isolated, you can call the function as many times as you like without any risk of packet collisions.
Promises
Since the callbackified NoFlo graph looks like a typical Node.js function, you can use all the normal flow control mechanisms like Promises or async with it.
For example, you can convert it to a Promise either manually or with a library like Bluebird:
// Load Bluebird
var Promise = require('bluebird');
// Convert the wrapped function into a Promise
var promisedGraph = Promise.promisify(wrappedGraph);
// Run it
promisedGraph({
in: 'baz'
})
.then (function (result) {
console.log(result.out);
});
This functionality is available right now on NPM.
[Less]
|
After several months of work, NoFlo 0.8 is finally out as a stable release. This release is important in that it brings the Process API for NoFlo components to general availability, paving way for the 1.x series.
Process API
We introduced Process
... [More]
API in the 0.7 series last summer, but at that stage it wasn’t deemed good enough for production use. There were issues with features like bracket forwarding, as well as API convenience questions that we’ve since tackled. With NoFlo 0.8 onwards, Process API is the recommended way to write all components.
Here are some of the major features coming to Process API with 0.8:
hasData and getData methods for dealing with data packets in the inports
hasStream and getStream methods for dealing with complete streams of IPs in the inports
Calling output.done is now required to signal when component is done processing
Automatic bracket forwarding from and to multiple ports as defined in forwardBrackets configuration
Support for receiving packets from addressable ports using the [portname, idx] syntax
More information on how to use the API for writing NoFlo components can be found from the Process API guide I wrote back in January.
Big thanks yet again to Vladimir Sibirov for his work pushing Process API to production!
WirePattern
Before Process API, WirePattern was the preferred convenience method used for writing NoFlo components. It made it easy to synchronize data from multiple inports, and to manage lifecycle of asynchronous components. Process API was largely designed to address the learnings we’ve had from the years using WirePattern in production, both the conveniences and the pitfalls.
Since vast majority of current open source NoFlo components use WirePattern, it wasn’t feasible to simply go and deprecate the API. Instead, what we did in the 0.8 cycle was port WirePattern to actually run on top of Process API.
So, when you update to NoFlo 0.8, all WirePattern components will automatically switch to using Process API internally. This means that existing components and graphs should stay working as they always did, except now fully compatible with features like scope isolation, bracket forwarding, and the new network lifecycle.
In 0.8 series we still also ship the original WirePattern implementation, which can be enabled in two ways:
Passing a legacy: true as an option to the WirePattern function. This will cause that component to use the legacy WirePattern iimplementation
Setting NOFLO_WIREPATTERN_LEGACY environment variable to force all WirePattern components to use legacy mode
There should be no reason to use the legacy mode. If you find a backwards compatibility issue forcing you to do so in your projects, please file an issue.
Component and network lifecycle
Another area of focus for 0.8 was the network lifecycle. The legacy component API in NoFlo had no way for components to tell when they’re done processing data, and hence the network wasn’t able to determine accurately when it was finished.
With Process API we give components much better way to handle this — when you’re done processing, call output.done(). Until then the component is expected to be doing work. When all components have deactivated, the network is considered finished:
To support the lifecycle better, we also made both component and network star-up and shutdown asynchronous with callbacks. This ensures every node in a NoFlo network can do everything it needs to initialize or clean up at the right stage of the flow.
The network methods are:
network.start(callback) to start the network. This starts all components and sends the Initial Information Packets
network.stop(callback) to stop the network. This calls all components to shut down, closes connections between them, and clears any in-flight inport buffers
If your component needs to do anything special at start-up or shutdown, the new methods it can provide are:
component.setUp(callback) called when network starts. Component can do self-initialization but should not send anything at this stage
component.tearDown(callback) called when network stops. Stateful components can clean their state at this point, and generators should stop generating (remove event listeners, shut down socket connections, etc)
The tearDown method replaces the old shutdown method. While shutdown method still gets called, all components should migrate to the async-capable tearDown in this release cycle.
Deprecated features
The 0.8 series adds deprecation warnings to features that will be removed from the eventual NoFlo 1.x release. You can find a full list from the ChangeLog, but here are some notable ones:
Synchronous WirePattern usage (switch to async: true or port to Process API)
noflo.AsyncComponent and noflo.helpers.MapComponent (should be ported to Process API)
noflo.InPort legacy packet methods process, receive and contains
Legacy noflo.Port and noflo.ArrayPort implementations. Use the modern noflo.InPort and noflo.OutPort instead
component.error and component.fail methods. Use WirePattern or Process API error callback instead
By default using a deprecated feature only logs a warning. If you want to make these fatal, set the NOFLO_FATAL_DEPRECATED environment variable. This can be useful for Continuous Integration.
Getting 0.8
NoFlo 0.8.x releases can be found from NPM.
For Node.js projects, the noflo-nodejs command-line runner has been updated to 0.8
For browser projects, noflo-browser-app scaffolding has been updated to 0.8
As I write this, the hosted browser runtime is still on 0.7. We will hopefully get it updated shortly.
[Less]
|
