Salesforce-Node.JS integration with Streaming API
Streaming API - Quick Intro
The Force.com Streaming API can be used to expose a near real-time stream of data from the Salesforce org in a secure and scalable way. The Force.com Streaming API utilizes a publish/subscribe model where we can create one or more named topics, each of which is associated with a SOQL query. External applications can then subscribe to one or more of these topics, using the Bayeux protocol. As and when the relevant data is created or updated, the published query is re-evaluated, and in case there is a change that alters the results of the query, a notification is published to all the applications that have subscribed to the topic.
Why Streaming API ?
This is an obvious question. Why Streaming API, when we already have REST and SOAP API. Well, the answer is simple - Callout Limits and overhead due to continuous polling to external applications. Apex Callouts and Outbound Messaging are the most preferred ways to integrate salesforce with external system. But, there will be times when one reaches their Callout Limits or want to use Outbound Messaging with non-SOAP endpoints.
If your application is currently polling the SOAP or REST API to detect changes, do consider using the Streaming API; and it will lead to better performance with lower resource consumption.
Limitations - Message Reliability
Streaming API doesn’t guarantee durability and reliable delivery of notifications. Streaming servers don’t maintain any client state and don’t keep track of what’s delivered. The client may not receive messages for a variety of reasons, including:
- When a client first subscribes or reconnects, it doesn’t receive messages that were processed while it wasn’t subscribed to the channel.
- If a client disconnects and starts a new handshake, it may be working with a different application server, so it receives only new messages from that point on.
- Some events may be dropped if the system is being heavily used.
- If an application server is stopped, all the messages being processed but not yet sent are lost. Any clients connected to that application server are disconnected. To receive notifications, the client must reconnect and subscribe to the topic channel.
To get a glimpse of what exactly we are trying to build here you can watch the live demo of the integration by following the below steps:
- Open this link which is a Salesforce site I have created to let the guest users create Account in my org without any credentials.
- Next open
this link(Node.JS app created on OpenShift) side-by-side the above window.
- Now create an account in the first window and see the same appearing in the second window in real-time (magic!).
What is Nodejs?
Watch this superb video by Pedro to understand Nodejs; its key features and what one shouldn't expect from Nodejs.
Create a Connected App
- Login to your Salesforce Org and in Setup go to Apps.
- Scroll down and under Connected app section Click New to create a connected app.
- Enter the details as shown below replacing the values relevant to your org
Caution: OAuth Scopes is selected as Full Access(full) for demo purpose. Make sure you completely understand the repercussion before applying one
- Save and now you should be available with the Consumer Key and Consumer Secret which will be required in Node.JS configuration to connect to Salesforce.
Now create a Push topic by executing the below code in the Developer Console. Alternatively you can create the same in Workbench.
PushTopic pTopic = new PushTopic(); pTopic.apiversion = 24.0; pTopic.Name = 'AccountsTopic'; pTopic.Description = 'New Account records'; pTopic.query = 'SELECT Id, Name FROM Account'; insert pTopic; System.debug('Push Topic created with ID: '+ pTopic.Id);
Check the below line in log to confirm the Topic creation
- Download the zip or clone the github repository here.
npm installon the prompt to install all the dependencies defined in
- Once all the dependencies are installed successfully open the
config.jsfile and replace the values for authentication fields as mentioned in the comments
In case you are not available with security token for your org then you can obtain the same in your My Personal Settings : Reset My Security Token. But take caution if you are already using the token somewhere else.
- Go to the folder where you have the Node code and run
node app.jsin a prompt in that location. If everything is properly configured then you will see the below message.
- Open the URL:
http://localhost:3001in a browser. Create an account in your org and you could see the same reflecting in real time in the browser (Thanks to Socket.IO).
Watch the video below to see the demo.