Caching data

The translator environment is built on the assumption that the transformation of data is happening in a fraction of a second with a linear flow of data coming in and being transformed out into a destination.

That allowed us to build the translator IDE so that we execute the script from start to finish with every change and generate all the useful auto-completion and annotations that make the environment really convenient for doing integration.

Web services do throw a little bit of a spanner in the works – even the fastest of them can have a few seconds of latency. That slows down the speed at which the translator can generate auto-completion and annotations. It makes for a much less smooth and convenient experience.

This is where a bit of caching can give a big benefit. The store module makes it easy to cache the results coming back from HTTP calls. The improvement in usability is spectacular. In my salesforce.com adapter I wrote some helper functions to wrap around the HTTP calls to cache the authorization token and the results of HTTP GET queries.

Here’s a couple of examples, the first is caching of the access token:

And the next example shows caching for a general HTTP GET:

Notice how the caching only happens within the context of the editor? This is when we need it – as we are writing the code and making use of auto-completion and annotations. Caching is switched off when we run the code in production.