Adaptable IoT

On May 24, 2017 I gave a short partner session at Techorama, a technology event in Belgium for both developers and IT Pros. You can find the slides on SlideShare:

Since it was a short session and a short slide deck, this post provides a bit more background information.

First, what do I mean with Adaptable IoT? Basically, an IoT solution should be adaptable at two levels:

  1. The IoT platform: use a platform that can be easily adapted to new conditions such as changed business needs or higher scaling requirements; a platform that allows you to plug in new services
  2. The application you write on the platform: use a flexible architecture that can easily be changed according to changing business needs; and no, that does not mean you have to use microservices

The presentation mainly focuses on the first point, which deals with the platform aspects that should be adaptable end-to-end at the following levels:

  • Devices and edge: devices should not be isolated in the field which means you should provide a two-way communication channel, a way to update firmware and write robust device code as a base requirement
  • Ingestion and management: with most platforms, the service used for ingestion of telemetry also provides management
  • Processing: the platform should be easy to extend with extra processing steps with limited impact on the existing processing pipeline
  • Storage: the platform should provide flexible storage options for both structured and unstructured data
  • Analytics: the platform should provide both descriptive and predictive analytics options that can be used to answer relevant business questions

Before continuing, note that this post focuses on Microsoft Azure with its Azure IoT Suite. The concepts laid out in this post can apply to other platforms as well!

Devices and Edge

There is a lot to say about devices and edge. What we see in the field is that most tend to think that the devices are the easy part. In fact, devices tend to be the most difficult part in an end-to-end IoT solution. Prototyping is easy because you can skip many of the hard parts you encounter in production:

  • Use Arduino or platforms such as particle.io: they are easy to use but do not give you full access to the underlying hardware and speed might be an issue
  • To demonstrate that it works, you can use simple and cheap sensors. But do they work in the long run? What about calibration?
  • You can use any library you find on the net but stability and accuracy might be an issue in production and even in the prototyping phase!
  • You can store secrets to connect to your back-end application directly in the sketch. In production however, you will need to store them securely.
  • Using TLS for secure connections is easy, provided the hardware and libraries support it. But what about certificate checks and expiry of root and leaf certificates?
  • You can just use WiFi because it is easy and convenient.

When you move to production and you want to create truly adaptable devices, you will need to think about several things:

  • Drop Arduino and move to C/C++ directly on the metal; heck, maybe you even have to throw in some assembler depending on the use case (though I hope not!); your focus should be on stability, speed and power usage.
  • Provide two-way communications so that devices can send telemetry and status messages to the back-end and the back-end can send messages back.
  • Make sure you can send messages to groups of devices (e.g. based on some query)
  • Provide a firmware update mechanism. Easier said than done!
  • Make sure the device is secure. Store secrets in a crypto chip.
  • Use stable and supported libraries such as the Azure IoT device SDK for C

Take into account that many devices will not be able to connect to your back-end directly, requiring a gateway at the edge. The edge should be adaptable as well, with options to do edge processing beyond merely relaying messages. What are some of those additional edge features?

  • Inference based on a machine learning algorithm trained in the cloud (e.g. anomaly detection)
  • Aggregation of data (e.g. stream processing with windowing)
  • Launch compute tasks based on conditions (e.g. launch an Azure Function when an anomaly is detected)

Ideally, the edge components are developed and tested in the cloud and then exported to the edge. Azure IoT Edge provides that functionality and uses containers to encapsulate the functionality described above.

Ingestion and management

The central service in the Azure IoT Suite for ingestion and management is Azure IoT Hub. It is highly scalable and makes your IoT solution adaptable by providing configuration and reporting mechanisms for devices. The figure below illustrates what is possible:

iothub

Device Twin functionality provides you with several options to make the solution adaptable and highly configurable:

  • From the back-end, you set desired properties that your devices can pick up. For instance, set a reporting interval to instruct the device to send telemetry more often
  • From the device, you send reported properties like battery status or available memory so you can act accordingly (e.g. send the user an alert to charge the device)
  • From the back-end, set tags to group devices (e.g. set the device location such as building, floor, room, etc…)

In a previous post, I already talked about setting desired properties with Device Twins and that today, you need to use the MQTT protocol to make this work. You can use the MQTT protocol directly or as part of one of the Azure Device SDKs where the protocol can simply be set as configuration.

The concept of jobs makes the solution even more adaptable since desired properties can be set on a group of devices using a query. By creating a query like ‘all devices where tag.building=buildingX’, you can set a desired property like the reporting interval on hundreds of devices at once.

Processing

The selected cloud platform should allow you to create an adaptable processing pipeline. With IoT Hub, the telemetry is made available to downstream components with a multi-consumer queue. An example is shown below:

processing

It is relatively easy to plug in new downstream components or modiy components. As an example, Microsoft recently made Time Series Insights available that uses an IoT Hub or an Event Hub as input. In a recent blogpost, I already described that service. Even if you already have an existing pipeline, it is simple to plug in Time Series Insights and to start using it to analyze your data.

Microservices on Kubernetes: a simple example in Go

In the previous post, Getting started with Kubernetes on Azure, we talked about creating a Kubernetes cluster and deploying a couple of services. There are basically two services:

  • Data: a service that exposes an endpoint to pick up data for an IoT device; you call it with http://service_endpoint:8080/data/devicename
  • Device: a service that can be used by the Data API to check if a device exists; if the device exists you will see that in the response

When you call the Data service, it will call the Device service using gRPC, using HTTP as the transport protocol. You define the service using Protocol Buffers. gRPC works across languages and platforms, so I could have implemented each service using a different language like Go for the Device service and Node.js for the Data service. In this example, I decided to use Go in both cases and use Go Micro, a pluggable RPC framework for microservices. Go Micro uses gRPC and protocol buffers under the hood with changes specific to Go Micro.

Ok, enough with the talk, let’s take a look how it is done. The Device service is kept extremely simple for an abvious reason: I just started with Go Micro and then it is best to start with something simple. I do expect you know a bit of Go from here on out. All the code can be found at https://github.com/gbaeke/go-device.

Lets start with the definition of Protocol Buffers, found in proto/device.proto:

syntax = "proto3";

service DevSvc {
    rpc Get(DeviceName) returns (Device) {}
}

message DeviceName {
    string name = 1;
}

message Device {
    string name = 1;
    bool active = 2;
}

We define one RPC call here that expects a DeviceName message as input and returns a Device message. Simple enough but this does not get us very far. To actually use this in Go (or another supported language), we will generate some code from the above definition. You need a couple of things to do that though:

  • protoc compiler: download from Github  for your platform
  • protobuf plugins for code generation for Go Micro: run go get github.com/micro/protobuf/{proto,protoc-gen-go} (if you have issues, use 2 gets, one for proto and one for protoc-gen-go)

To actually compile the proto file, use the following command:

protoc --go_out=plugins=micro:. device.proto

That compiles device.proto to device.pb.go with help from the micro plugin. You can check the generated code here. Among other things, there are Go structs for the DeviceName and Device message plus several methods you can call on these structs such as Reset() and String().

Now for main.go! You’ll need several imports: for the generated code but also for the dependencies to build the service with Go Micro. If you check the code, you will also find the following import:

_ "github.com/micro/go-plugins/registry/kubernetes"

As stated above, Go Micro is a pluggable RPC framework. Out of the box, a microservice written with Go Micro will try to register itself with Consul on localhost for service discovery and configuration. We could run the Consul service in Kubernetes but Kubernetes supports service registration natively. Kubernetes support is something you add with the import above. That is not enough though! You still need to tell Go Micro to use Kubernetes as the registry, either with the —registry command line parameter or with an environment variable MICRO_REGISTRY. Check https://github.com/gbaeke/go-device/blob/master/go-device-dep.yaml file where that environment variable is set. Besides Consul and Kubernetes, there are other alternatives. One of them is multicast DNS (mdns) which is handy when you are testing services on your local machine and you don’t have something like Consul running.

If you want to check the information that is registered, you can do the following (after running kubectl proxy --port=8080):

curl http://localhost:8080/api/v1/pods | grep micro

Each pod will have an annotation with key micro.mu/service-<servicename> with information about the service such as its name, IP address, port, and much more.

Now really over to main.go, which is pretty self explanatory. There’s a struct called DevSvc which has a field called devs which holds the map of strings to Device structs. The DevSvc actually defines the service and you write the RPC calls as methods of that struct. Check out the following code snippet:

// DevSvc defines the service
type DevSvc struct {
	devs map[string]*device.Device
}
func (d *DevSvc) Get(ctx context.Context, req *device.DeviceName, rsp *device.Device) error {
	device, ok := d.devs[req.Name]
	if !ok {
		fmt.Println("Device does not exist")
		return nil
	}

	fmt.Println("Will respond with ", device)

	// this also works
	rsp.Name = device.Name
	rsp.Active = device.Active

	return nil
}

The Get function implements what was defined in the .proto file earlier and uses pointers to a DeviceName struct as input and a pointer to a Device struct as output. The code itself is of course trivial and just looks up a device in the map and returns it with rsp.

Of course, this handler needs to be registered and this happens in the main() function (besides setting up the service and implementing a custom flag):

// register handler and initialise devs map with a list of devices
device.RegisterDevSvcHandler(service.Server(), &DevSvc{devs: LoadDevices()})

If you want to test the service and call it (e.g. on the local machine) then clone the repository (or get it) and run the server as follows:

go run main.go --registry=mdns

In another terminal, run:

go run main.go --registry=mdns --run_client

When you run the code with the run_client option, the runClient function is called which looks like:

func runClient(service micro.Service) {
	// Create new client to call DevSvc service
	DevClient := device.NewDevSvcClient("go.micro.srv.device", service.Client())

	// Call Get to get a device
	rsp, err := DevClient.Get(context.TODO(), &device.DeviceName{Name: "device2"})
	if err != nil {
		fmt.Println(err)
		return
	}

	// Print response
	fmt.Println("Response: ", rsp)
}

This again shows the power of using a framework like Go Micro: you create a client for the DevSvc service and then simply perform the remote procedure call with the Get method, passing in a DeviceName struct with the Name field set to the device you want to check. The client uses the service registry to know where and how to connect. All the serialization and deserialization is handled for you as well using protocol buffers.

So great, you now have a little bit more information about the Device service and you know how to deploy it to Kubernetes. In another post, we’ll see how the Data service works and explore some other options to write that service.

Using Azure Cognitive Services from Logic Apps

Azure’s Cognitive Services are very easy to use from within your own applications or more “code-less” solutions such as Azure Logic Apps. In this post, I will show a simple example of a Logic App that does sentiment analysis on incoming tweets. When the sentiment score is very high, an SMS is sent.

To perform sentiment analysis, use the Text Analytics API from Cognitive Services. First, in the Azure Portal, create a Cognitive Services account of type Text Analytics. After that account has been created, you will need the following information to be used in Logic Apps:

  • Endpoint: https://westus.api.cognitive.microsoft.com/text/analytics/v2.0
  • Key: use one of the two secret keys to access the API

If you create a Cognitive Services account in the free tier, you can make 5000 calls per 30 days.

Now create a Logic App in the Azure Portal and go to Triggers and Actions to enter the designer. I will not provide step-by-step details as working with triggers and actions in the graphical designer is very easy. To obtain the results we want, we will have to switch to Code View though.

First, from the Microsoft Managed APIs, add a Twitter trigger. Provide your credentials to Twitter and provide a search term.

Now click the + icon to add an action. To call the Sentiment Analysis API, use the HTTP action and provide the following information (Note: forgetting to specify the specific API to call is a common error; for the URI below also add /sentiment to perform sentiment analysis!!!):

2016-04-22 15_48_34-Logic Apps Designer - Microsoft Azure

Naturally, replace your key with one of the keys obtained from your Cognitive Services account. The body of your HTTP POST can be an array of documents, with each document having an id and the text you want to analyze. In our case, we want to analyze the Tweet text so we use the graphical designer to insert it. In Code View, this will be:

2016-04-22 15_52_59-Logic Apps Designer - Microsoft Azure

Now we want to send an SMS when the sentiment of the Tweet is very positive. The sentiment is expressed as a value between 0 and 1 where 1 is a really, really, really positive tweet.

To send an SMS when the sentiment is above 0.95, first click the + icon and add a condition. The value to evaluate is part of the HTTP body of the previous action. So add that and select greater than or equals and enter 0.95 in the value. Then switch to advanced view to see the expression you built. It will look like:

@greaterOrEquals(body(‘Http’), 0.95)

The above is not going to cut it though since the response body is JSON and the sentiment score needs to be extracted from it. Change the expression to the following:

@greaterOrEquals(float(json(string(body(‘Http’))).documents[0].score), 0.95)

Since the response body is an array of documents and we only have one document, just obtain the score from the first document.

2016-04-22 16_03_05-Logic Apps Designer - Microsoft Azure

Now we can click Add an action in the If yes section to send an SMS. You can use the Twilio Send Message Managed API to do so but you will need an account at Twilio for this to work. Alternatively, you can send an e-mail or just post a result to http://requestb.in. For Twilio, you will end up with something like below. Phone numbers have been blurred to protect the innocent.

2016-04-22 16_08_08-Logic Apps Designer - Microsoft Azure

In the above, we only want to show the score for the Tweet and not the whole body. This can be done in Code View:

In the Send_Message action, change the following:

@{body(‘Http’)} for @{triggerBody()[‘TweetText’]}

to:

@{json(string(body(‘Http’))).documents[0].score} for @{triggerBody()[‘TweetText’]}

Note that changes like the above can make the UI designer unavailable.

When you save this Logic App, incoming tweets containing Azure should be analyzed and you should get SMSs when tweets are very positive. Hey, it’s Azure, why shouldn’t they be? 🙂

You can check if the Logic App is executing correctly from the Operations tile:

2016-04-22 16_23_11-Logic Apps Designer - Microsoft Azure

For a search term like Azure, I recommend to turn off the Logic App if you don’t want to exhaust your 5000 free tier API calls.

Windows Azure Point-to-Site Networking

If you are having trouble with the point-to-site VPN configuration in Windows Azure, here are some tips about the procedure:

  • Follow the procedure located at http://msdn.microsoft.com/en-us/library/windowsazure/dn133792.aspx for creating the virtual network and the gateway.
  • When configuring the certificates for the VPN connection, first create the self-signed root certificate with the following command:  

    makecert -sky exchange -r -n "CN=RootCertificateName" –pe -a sha1 -len 2048 -ss My

  • The above command creates a self-signed root certificate and stores it in your certificate store (Certificates – Current User\Personal\Certificates). Next, export that certificate to a .cer file and upload it to Azure from the dashboard of the virtual network using the Upload client certificate link (the name of that link will probably be changed in the future Smile) I also stored the root certificate in my Trusted Roots.
  • Now create a client certificate with the self-signed root certificate as the issuer. The command I used is different from the one in the documentation because it did not work for me. I used:

    makecert -n "CN=ClientCertificateName" -pe -sky exchange -m 96 -ss
    my -a sha1 -is my -in "RootCertificateName"

  • The above command creates the client certificate in the same store as the root certificate and uses the root certificate previously generated as the issuer. Be sure to check that the issuer is the root certificate you uploaded to Azure.

In the dashboard of the virtual network, download the x64 or x86 client VPN package and install it. There will be an extra network connection that uses SSTP to connect to your Azure gateway:

image

 

In Azure the dashboard should show connected clients:

image