Selfhosted integration systems overview

Integration systems could be Commercial with a lot of paid connection modules and a bit of free ones; Or kind of free but with some strange license; Or ready for self-hosting and not.

There is a lot of integration systems available…

Use cases

Could be good for

• organising your own integration between different [social network] platforms,
• configuring notifications on IoT events, or
• setting up full-fledged POSSE system.

If you curious what a POSSE is… POSSE is an abbreviation for Publish (on your) Own Site, Syndicate Elsewhere, the practice of posting content on your own site first, then publishing copies or sharing links to third parties (like social media silos) with original post links to provide viewers a path to directly interacting with your content.

And IoT is an Internet of Things, just in case you’ve forgotten.

Cloud-based

Universal platforms

Like AWS with SNS, SQS, EventBridge, AppFlow and others

Tailored

Plenty of choice

• Zapier
• Make
• Prefect
• n8n
• Jitsu

and others.

Quite convenient but could be pricey.

Self-hosted integration systems

Apache

Apache has many projects and in integration niche have a look at Nifi

Mulesoft’s ESB

This Mule ESB with studio and some other tools are forming platform with nice set of Connectors.

https://www.mulesoft.com/platform/studio

Keeping it here. Might be overkill for a personal project.

Can be downloaded here: https://www.mulesoft.com/lp/dl/anypoint-mule-studio

Check the license.

Huginn

Implemented in rubyonrails

https://github.com/huginn/huginn/blob/master/doc/docker/install.md

One of the most mature open source integration systems.

Activepieces

Implemented in Javascript

https://www.activepieces.com/docs/contributing/overview

node-RED

https://nodered.org/docs/tutorials/first-flow

Awesome integration platform with quite low hardware requirements.

Interface is industry-style.

Custom connections are coded with JavaScript.

Windmill

https://github.com/javipalanca/spade

Smart Python Agent Development Environment

A multi-agent systems platform written in Python and based on instant messaging (XMPP).

Develop agents that can chat both with other agents and humans.

MIT license

StackStorm

https://stackstorm.com/

from their site:

StackStorm is a platform for integration and automation across services and tools…

StackStorm helps automate common operational patterns. Some examples are:

• Facilitated Troubleshooting - triggering on system failures captured by Nagios, Sensu, New Relic and other monitoring systems, running a series of diagnostic checks on physical nodes, OpenStack or Amazon instances, and application components, and posting results to a shared communication context, like Slack or JIRA.
• Automated remediation - identifying and verifying hardware failure on OpenStack compute node, properly evacuating instances and emailing admins about potential downtime, but if anything goes wrong - freezing the workflow and calling PagerDuty to wake up a human.
• Continuous deployment - build and test with Jenkins, provision a new AWS cluster, turn on some traffic with the load balancer, and roll-forward or roll-back, based on NewRelic app performance data.

Luigi

https://luigi.readthedocs.io/en/stable/

Luigi is a Python package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more.

BeeHive

https://github.com/muesli/beehive

A flexible event/agent & automation system, which allows to create your own agents that perform automated tasks triggered by events and filters. It is modular, flexible and really easy to extend for anyone. It has modules (we call them Hives), so it can interface with, talk to, or retrieve information from Twitter, Tumblr, Email, IRC, Jabber, RSS, Jenkins, Hue - to name just a few.

List of integration systems

Apache-2.0 License

System Name	Implementation Language
airflow	Python
luigi	Python
mage	Python
node-red	Javascript/node.js
prefect	Python
taskflow	Python
StackStorm	Python
syndesis	Java
thingsboard	Java

MIT License

System Name	Implementation Language
actionsflow	Javascript/Typescript
active_workflow	Ruby
huginn	Ruby
jitsu	Javascript
novu	Javascript/Typescript
pydoit	Python

BSD License

System Name	Implementation Language
apprise	Python
flogo	Golang

*GPL License

System Name	Implementation Language
automatisch	Javascript/Typescript
beehive	Golang
spiffworkflow	Python

Other licenses (check trhem first!)

System Name	Implementation Language
activepieces	Javascript/Typescript
chainjet	Javascript/Typescript
n8n	Javascript/Typescript

My take

I like the node-RED the most.

• Good set of availiable connectors
• Low system requirements (like Raspberry PI?)
• Easy coding with JS
• BUT See for yourselves, the learning curve could be steep depending on your exp.