Kloudless Walkthrough: Building with our Open-Source Authenticator JS Library

This post was written by our engineer Chris Prochnow.

In this walkthrough, we’ll use the Kloudless open-source Authenticator JS library that prompts the user to grant access to a cloud service. We will look at the set-up of a basic web server, and explore how to use the Authenticator to set up a client-side form with all the data needed to send to the server-side for API requests.

We’ve written Authenticator.JS documentation that we will reference through the article. You can find it on GitHub here.

Getting started

Here is a form with all the fields needed to create a basic text file, which will eventually be stored in the Google Drive account that the user has been granted permission to access.


<form action="/post" method="post">
    <input type="text" name="fileName" placeholder="File Name">
    <textarea name="fileBody&quot; rows="7">File text here.</textarea>
    <button type="submit">Create and upload file</button>
  <input type="hidden" name="access_token" id="access_token" value="">
  <input type="hidden" name="account_id" id="account_id" value="">

Our web server is based on Node and Express. We will use the submitted form data to create a file, verify the authentication token, construct an API call, and run some cleanup once the action is complete. If you look at the code above, there are two inputs with the type=“hidden” that do not yet contain values. The values account_id and access_token must be supplied to us in order to access the storage service. Here is where the Authenticator JS library comes in.

First, we must obtain the correct values. To do so, we’ll need to embed the Authenticator in the client-side page using this script tag, which is documented in the Authenticator’s README here.



This exposes a global Kloudless object, which contains the  Kloudless.authenticator method. This method is what we’ll use to trigger an authentication pop-up for the end-user:

Kloudless.authenticator(element, params, callback)

Before that can happen, we need to set up some logic to handle the end-user’s input: 

  • element is the triggering action. For this example, I’ve used jQuery to set the triggering element.
  • We also need to pass the configuration to the Kloudless.authenticator method to suit our application needs. These are required parameters that the end-user does not need to be aware of, but that do need to be passed in to the authentication.
  • At the minimum, you must pass the Kloudless application ID to identify the Kloudless app. This is provided as the client_id.

In our case, we only need to authenticate one service, Google Drive, so we will also be setting the scope to 'gdrive' in our configuration. The config will look like this:

  const config = {
      'client_id': ‘[YOUR CLIENT ID HERE]’,
      scope: 'gdrive',

This gives a great setup for the authenticator.js!

Let’s see how what we have so far could be laid out:

$(document).ready(() => {
  const config = {
      'client_id': ‘[YOUR CLIENT ID HERE]’,
      ‘scope’: 'gdrive',
  let auth = Kloudless.authenticator(press, config, callback);

Pairing data with input sources

The next pieces we need to include will be adding additional JS to set our received data to the correct inputs in our form, and setting a callback function that will determine what we do with the response data. The JS setup is pretty quick and involves defining some variables required using jQuery.

const $press = $('#submitAuth');
const $account_id = $('#account_id');
const $access_token = $('#access_token');
const $uploadButton = $('#uploadButton');

Callback function

Now for the callback function setup. We’ll need to handle errors, as well as decide what we’ll do with our successful data. Using the variables we just set up, we can reset the input values with the returned information:

let callback = function (result) {
   if (result.error) {
     console.log('An error occurred:', result.error);


Our authenticator.js should now look something like this:

$(document).ready(() => {

 const config = {
     'client_id': 'NyRVsMqKtYS7g9t40WuSqhJCAxi4ANZ3p6MOsKfCFjXX_xcb',
     'scope': 'gdrive',

 const $press = $('#submitAuth');
 const $account_id = $('#account_id');
 const $access_token = $('#access_token');
 const $uploadButton = $('#uploadButton');

 let callback = function (result) {
    if (result.error) {
      console.log('An error occurred:', result.error);



  let auth = Kloudless.authenticator($press, config, callback);


With that, we now have all the data needed to send to our server-side code to make an API call. This initial set-up uses an OAuth Kloudless key, which will be sufficient for basic development needs, but will be subject to rate-limit restrictions.

Setting up your own OAuth credentials

The next part of this blog will discuss how you can set up your own OAuth credentials. We’ll continue to use Google Drive as our example. 

Setting up these credentials provides several benefits: it allows you to manage your own tokens, implement a better branding experience, and exempt you from sharing the rate limits/quotas that Kloudless default keys are subject to. So let’s dive in!

To start off, navigate to the OAuth Credentials page in the developer portal for your app. You can find this page under Security -> Credentials in the developer portal. Be sure to select the correct app from the dropdown menu in the top left corner of the page.

You should see a list of cloud storage services that looks something like this:


In our example, we are looking for the listing for Google Drive. When you find that entry, you will see that Google Drive is by default set to the Default Kloudless keys.

If you have existing accounts connected to the Default key, you will see a message denoting the number of accounts using that key, and an option to delete these accounts bordered in red, with some additional information.


We do not need these accounts to continue to have access, and would like to have them re-authenticate through our app in the future. In this case I will check the box to have the existing accounts deleted.

Please note: this deletion does not affect the contents of your account or delete the account, but they do remove access to the account the keys are associated with.

Once you click ‘Save’, the first thing you’ll see is several new fields with instructions and helpful information specific to OAuth tokens and Google.


Keep this browser tab open—we’ll be coming back to it in a bit.

We’re now ready to head over to the Google Developers Console!

If you haven’t yet, create a new Google project. This is where you’ll manage apps and services that request access to your users’ Google accounts, and generate a Client ID/Consumer Key and our Client Secret/Consumer Secret.

Once that is done, there are a few steps to follow to get the information from the Google project. It should look something like the image below once you have created the project, with your project name in place of ‘kloudless test auth’:


Our next step is to create the credentials needed for the Kloudless app. Click on the ‘Credentials’ tab to look at our options for creating credentials. In this case, we’re creating an OAuth Client ID, so select that option and then select the correct application type. In this case, I selected ‘Web application’.

All right! We now have a Client ID/Consumer Key and our Client Secret/Consumer Secret.

We’ve got something else to edit in the Google Developers Console: the callback URL or Redirect URL. Refer back to the Kloudless Credentials page and grab the link supplied there. Copy and paste that into the field under ‘authorized redirect URIs’, and click save.


Note: keep in mind that the Client secret is just that—a SECRET. Be mindful of where you save it, and do not leave it exposed in your documentation.

Finally, head back to the Kloudless Credentials page. Enter the Client ID and Client secret into their respective fields, and save the app.

That’s it—you’ve successfully moved your app from using the default Kloudless OAuth keys to your own OAuth keys! 😀🎉


We hope that you found our introduction to the Kloudless Authenticator JS library useful. As we grow, we’ll be expanding our open-source initiatives—and the accompanying guides, walkthroughs, and other materials—so stay tuned for more posts like this one.

Want to build on these initiatives? Come join us—we’re hiring across a ton of different functions in our Berkeley, CA and Taipei, TW offices!

Lessons I’ve learned deploying SaaS software on-prem


If the best way to learn something is to teach it, then I’ve learned a surprising amount by guiding customers through on-premises deployments of our SaaS application.

Dealing with flaky DB migrations on Windows machines is a far cry from operating our fine-tuned AWS VPC. I’m glad we bit the bullet and deployed on-prem early on—it’s shaped our application and system design decisions in ways I couldn’t have imagined.


We launched our integrations platform, Kloudless, in 2014 as a multi-tenant web app on AWS. As we began to work with our first enterprise customers, we quickly realized that our SaaS application didn’t come close to meeting the various expected regulatory compliance and audit requirements. Several organizations simply couldn’t use Kloudless until we certified compliance with PCI and HIPAA, or underwent a SOC 2 audit.

As CTO, I challenged my team to find a solution. We soon introduced Kloudless Enterprise, the on-premises version of our cloud solution, to allow our customers to run Kloudless as part of their on-prem stack. Today, Kloudless Enterprise Docker containers, AMIs, and OVAs power applications used by Fortune 50 companies and some of the nation’s largest banks. This year, we crossed 500M upstream API requests a month.

Here are some of the major lessons I’ve learned along the way.

Kloudless Enterprise (source)
\==============_=_/ ____.---'---`---._____
             \_ \    \----._________.----/
               \ \   /  /    `-_-'
          /____          ||


Deploying is hard, so make it as easy as possible

Kloudless Enterprise is installed and updated often, so our customers require this process to be completed quickly. Limiting the number of exposed moving parts exposed enables us to provide updates that are as simple as deploying new Docker containers. Developers can start out using local PostgreSQL and Redis services and configure external ones later for scalable production deployments. This structure is similar to GitLab’s Omnibus packages, which are a great example.

We tied licenses to individual release channels for images and containers since a patch release is sometimes required for specific customers or a specific version. Secret: We also provide Kloudless operators the ability to update individual application code packages to the next patch release within a container, immutability be damned. While this is an anti-pattern and is definitely not the recommended option, it’s occasionally proven to be the path of least resistance in dealing with necessary change management review processes.

Preserving all application state outside of the container, like licenses, encryption keys, and meta-configuration, allows updates to be seamless. Using external storage, such as Docker volumes and network-attached storage like EBS, preserves configuration during updates.

You will thank your support tooling

SLAs for Enterprise and on-prem deployments often require the ability to quickly diagnose issues, ideally with self-serve tools like command-line utilities and monitoring integrations. Even so, our engineering team has had to dive into remotely deployed Kloudless development clusters to walk operators through configuration and diagnose hard-to-reproduce issues. Making this as easy as troubleshooting any cloud environment is a major benefit. When things don’t go as planned, support staff need tools available to assist customers with.

We built command line utilities to do everything from reading and sending log data, to managing users, maintenance windows, and configuration updates. Our most frequently used tool authenticates an outbound connection from Kloudless Enterprise instances to a secure remote bastion in our network for engineers to assist via a reverse SSH tunnel. Definitely use autossh for this kind of stuff. This utility lets us access our customers’ instances behind firewalls at their request while also maintaining their compliance requirements.

Scaling is harder when everyone does it their own way

Private Kloudless deployments reach upwards of several machines running 100+ cores combined. You can prevent unnecessary stress down the line by providing canonical scaling guides and items to consider.

Here are some issues we’ve seen while deploying privately hosted instances:

  • Postgres running out of disk due to insufficient capacity planning for activity stream data.
  • 10+ small AWS instances instead of 5 larger ones. Fun fact: AWS network throughput is based on instance vCPU, so adding more instances isn’t the answer to network bottlenecks!
  • Unhandled network partition issues. We quickly switched from a stateful, primary/secondary clustering format using etcd to a masterless model. This was not only simpler for our customers’ Ops teams, but turned out to be more robust and better able to handle large deployments.
  • Diverse deployment environments and tooling.
    Our initial version had an interactive setup process that required less preparation, but turned out to be a stumbling block for large production environments. Non-interactive deployment improves automation as well as integration with existing tools like AWS CloudFormation and Docker Compose.

Documentation makes things official

While on the topic of writing, I’ve noticed an entirely distinct set of documentation we’ve had to provide our on-prem customers. We expected to contribute API docs and information around setup, but we’ve also had to create far more documentation, including:

  • SOPs and security policy docs to provide security and legal teams
  • HA, scalability, and monitoring guides for operations
  • Solution walkthroughs and examples of configuration combinations for engineering
  • Detailed release notes and update paths
  • System troubleshooting guides

It’s important to be upfront about all data transmissions and to ensure tools initiating outbound connections are opt-in by default to not raise the ire of a customer’s security team.

The Application

On-prem vs. Cloud is best as a feature flag

The worst deployment strategy would be to fork the cloud application for on-prem deployments. In addition to increased maintenance costs, this results in bugs that dogfooding would have otherwise caught. Introducing a feature flag that indicates whether the application is run on-prem helps to easily toggle functionality like license restrictions and alternate web application UI.

Cloud to On-prem is an upgrade away

But wait, there’s more: using the same application and data stack makes it easy to transition cloud customers to private deployments or on-prem solutions without a complex data migration. While environment-specific items like encryption keys necessitate a documented upgrade procedure, it is possible to simplify the transition by limiting the deviation in application behavior. Plus, sales teams love the frictionless path to increasing revenue.

Configuration should stay as configuration

Ask the front-end design team to sit this one out. A friend of mine at GitHub contributes to GitHub Enterprise, which inspired a lot of our design choices when building out our own solution. We heeded one valuable bit of advice to reduce the overhead of configuration. We scrapped fancy settings dashboard web UIs and just provided a YAML file to edit. This saved us an inordinate amount of time shipping features. We’ve also:

  • had fewer questions than expected since the point-and-click folks stay away
  • accessed the full suite of YAML data structures to represent state
  • inherited configuration from org-wide, license-specific, and deployment-specific settings into each machine’s configuration
  • been able to access and manage configuration easily via utilities like Salt

Leverage open source

If you’d like a fun conversation with engineering and procurement teams at enterprises, then tell them they need to purchase even more on-prem software to actually view analytics, logs, monitoring, and more. Using third-party SaaS vendors works great for a cloud product, but on-prem software needs to come with batteries included for core functionality.

Fortunately, allowing a variety of open-source tools to be configured helps package functionality and prevent these concerns. For instance, Kloudless provides options to send system and application metrics to StatsD and InfluxDB as well as ship log data externally via standard protocols. This allows Kloudless to integrate with existing monitoring infrastructure with minimal effort.

Open source utilities also help users easily configure application security. For example, take advantage of Let’s Encrypt to provision SSL/TLS certificates for custom domain names quickly.

Application Security is as good as its documentation

Security is like a sine wave that peaks throughout this blog post at periodic intervals.

Application security goes beyond technical solutions that satisfy engineering. For example, summaries of compliance with certifications and items like pen test results are equally valuable. Security posture is relevant at every stage since it is one of the primary reasons for deploying on-prem. Providing a well-written, thorough information security policy covering everything from SDLC processes like code reviews, to information on internal controls, helps satisfy customers’ voracious need for security-related collateral.

“Enterprise” is an anagram for “Re-enter IPs”

We’ve had multiple requests for items like SSO, access controls for team members, activity logs, IP whitelists, and more. Making these easily configurable allows for frictionless deployments. However, we’ve noticed that these haven’t been a deal-breaker in most cases—we released Kloudless Enterprise without several features popular with larger customers. This could be specific to our type of application, but I’m glad we chose to ship our product ASAP instead of building features we assumed would be required. Check out enterpriseready.io for more details on features frequently included in software for enterprise customers.

Side-note: We also adjusted our build pipeline to compile and obfuscate application code from the very beginning to protect a different kind of IP, but in hindsight, it is debatable as to whether this was even necessary.

Technical solutions for licensing save time till they don’t

At the end of the day, a customer is likely purchasing some concept of a license “key” to run the application privately. We’ve found that the minimum effort required here is to build in validation checks and detail what data they transmit.

Avoid technical solutions for payments early on

Let’s say you create a self-hosted version of your product and you have a great billing system running on your cloud version—even using the latest Stripe Billing API—with a clear licensing model. I’ve found it’s still best to invoice customers manually until you’ve identified a repeatable process. We avoided prematurely optimizing payments with click-through solutions that would get in the way of customized pricing schemes, especially since the volume of enterprise deals is relatively low early on. Unlike in SaaS apps, it usually isn’t an option to disable accounts due to a Net 30 invoice delayed a few days. License terms also vary from quarterly and annual subscriptions to multi-year contracts, so I recommend setting aside the monthly cron job or equivalent solution.

We built an internal dashboard to update licenses entirely independently of payment, and we avoided directing customers purchasing on-prem solutions to the billing options provided to cloud customers. It’s hard enough to build in feature flags to deal with varied license requirements; requiring a single payment flow will simply slow things down.

APIs all the way down

Kloudless integrates APIs, but also provides a Meta API to manage the API integration platform. My goal this year is to convince our engineering team to integrate Kloudless with itself. In all seriousness, programmatic manipulation of the application is a massive benefit for engineering teams deploying the solution on-prem. It enables workflows that the customer may not have even bothered bringing up earlier, like requiring key rotation every 90 days.

I highly recommend an API that manages all data in the application, especially provisioning and deprovisioning workflows.

Would we do it again?

Yes, but I’d probably charge more right off the bat. 🙂

Ultimately, our GTM strategy incorporated deploying our application where the customer felt comfortable using it; either our multi-tenant cloud, a private deployment managed by us, or entirely self-hosted VMs and containers. A key decision to make for vendors considering on-prem deployments is simply whether the overhead described above is worth the benefits it brings.

Kloudless provides unified APIs to integrate applications with several software services in one go. We deploy where software services run, either in the cloud or on-prem. Sound interesting? We’re hiring!


Securing enterprise data across cloud services

We’re right around the corner from this year’s RSA Conference, held in San Francisco from April 16-20! As the conference approaches, it’s the perfect time for enterprises to evaluate their security priorities for the coming year.

What’s new in cyber security?

Interest in cyber security solutions has risen dramatically this decade. We know that the average enterprise is increasingly adopting online cloud-based SaaS tools for use in a variety of situations, from workflows and productivity to software development. But as data moves from traditional on-premise content repositories to the cloud, data becomes fragmented across a multitude of ecosystems. As a result, new technologies are emerging to protect data outside the safety of a corporate firewall.

Source: Stacy Jones from Fortune graphs Wade Baker’s analysis of RSA data.

Given the sensitive nature of security in the enterprise, it’s imperative for an enterprise’s security initiatives to align and adapt to a continuously evolving threat landscape. Yet it’s no ordinary feat to uniformly and reliably monitor activity on every disparate cloud service used by an organization.

Announcing new Kloudless Security APIs

When we launched the first Kloudless unified API a few years ago, we didn’t expect the variety of data security solutions that we would enable. Several of our users, customers, and partners are RSAC sponsors, where they showcase security solutions connecting to business applications like Office 365 and G Suite via Kloudless.

As it turns out, our unified APIs offer strong benefits designed for apps offering security solutions for cloud services, including, but not limited to:

  • Capturing information on all organization data in the cloud service, and who can access it;
  • Monitoring any and all activity occurring organization-wide;
  • Remediating threats using the Kloudless unified APIs for Storage, CRM, ITSM, and more;
  • Running Kloudless Docker containers on-premises to meet and exceed custom security and compliance requirements.

We’re dedicating more time this quarter to improve our security solutions with robust and relevant features that our users have been asking for. Today, we’re excited to announce two new versions of API endpoints targeted specifically for security integrations:

  • Unified Audit Events
    • Kloudless makes a stream of event data available in a consistent format. This includes information on any data being modified as well as behavioral events like logins, sharing, user/group creation, etc.  

  • Unified Permissions
    • Access controls are vital to any security company, and these controls can be extremely complex. The Kloudless v2 Permissions API provides granular information on the various access levels available on objects.

See you at RSA!

We’re looking to launch additional data security solutions in the upcoming months to complement the suite of v2 endpoints we released today. At the RSA Conference, we’ll be learning more about current trends, so we can include the most relevant features in our solutions.

Interested in meeting up at RSA? Drop us a line at hello@kloudless.com!   

Kloudless raises $6M to ease the software integration process

Website graphic

We’re thrilled to announce that we’ve raised $6M in Series A funding in a round led by Aspect Ventures, with participation by Bow Capital, Alibaba Taiwan Entrepreneur Fund, Heavybit, and Ajay Shah of Silver Lake Capital!

With its one-to-many approach, Kloudless Unified APIs free developers and product teams from building tedious and expensive one-off integrations. Try it out here.

At Kloudless, we have a firsthand understanding of how frustrating it can be to build the integrations we need. In our increasingly interconnected world, new SaaS services launch daily, and each one offers a unique API with its own documentation and features. It’s little wonder that API usage has exploded in the last few years.

In 2015, our team set out to change the way integrations were built by creating a developer-first solution to the problem—and in the process, became the first integrations platform to offer a solution tailored to the unique needs of developer and product teams. Our Unified APIs eliminate the redundancies of building integrations and now free thousands of developers to focus on their core product.

Since then, we’ve:

  • Grown from our small Berkeley-based team of 5 to 20 (and launched our first-ever international office in Taipei, Taiwan);
  • Welcomed Blackboard Inc., PlanGrid, ShieldX, OneIdentity, and other great companies to our platform;
  • Built up a strong community of over 15,000 developers from 4,000 different organizations.

We’re immensely proud of how far we’ve come and are equally excited to start the next chapter of the Kloudless journey!

What’s next?

Our new round of funding will help Kloudless to continually grow the success of our users, our platform, and our team.

Some of the technical improvements we’ve already planned include:

  • Boilerplate API integrations for any SaaS service’s HTTP API, with the help of automation
  • A new framework for workflow automation built on top of Kloudless’ extensive API platform
  • Expanding API unification and cross-connector capabilities like KQL
  • Additional production-ready UI tools to solve workflow use cases for companies in any industry

Of course, there’s more to come—so stay tuned!

Built by developers, for developers

Our new round of funding also makes it possible for us to make a concerted investment in initiatives that ensure the vibrancy and connectivity of the developer community.

As our users grow, we’ll create high-quality documentation, strengthen our SDK offerings, and cultivate responsive community forums to ensure that the experience of building with Kloudless continues to be efficient (and, dare we say, enjoyable!). Our efforts will focus on the most common use cases that our users encounter when integrating with other services.


We’re excited to tackle our road map and delight our users with new Kloudless platform expansions and features. To our existing Kloudless users, thank you for your support all these years – we wouldn’t have gotten here without you.

Haven’t tried Kloudless before? Sign up for a free account here to get started today!

We’re also hiring for a bunch of different functions, from Marketing to Sales to Engineering, in our Berkeley and our Taiwan offices. Check out our listings here!

About Kloudless

Kloudless’s unified APIs enable developers to code once to connect their applications to several software services instead of integrating with each API individually. Businesses use Kloudless’s unified API products to connect to many kinds of enterprise applications like Salesforce, SharePoint, Box, Dropbox, G Suite, Slack, ServiceNow and more.

Kloudless is trusted by more than 15,000 developers in over 4,000 organizations. To learn more, visit https://kloudless.com.

Introducing the Kloudless Query Language and new CRM connectors

Kloudless initially launched the unified CRM API with support for Salesforce and Microsoft Dynamics, enabling developers to connect apps with two large open SaaS CRM platforms. The initial release provided core features such as CRUD operations for all objects, near real-time activity monitoring, and making batch requests.

We often think of Salesforce, Microsoft, and SAP as the largest players in the space, but Gartner reports show that they only account for 35% of the CRM market share.

We are excited to announce support for a number of other CRM services in our unified CRM API, including Oracle Sales Cloud, SugarCRM, Hubspot CRM, Zoho CRM, and Pipeliner CRM. Existing customers using the unified CRM API can now offer these integrations to their users with minimal additional effort.

Here is a current breakdown of feature availability of the CRM API across the CRM services we support:

Salesforce Dynamics Oracle Pipeliner Zoho SugarCRM Hubspot
Search        ⏱
Events ✓*

: Supported            ✓*: Partially supported            ⏱: TBD

Kloudless Query Language (KQL)

We are also excited to announce a common query language to access object data in any supported CRM service.

The initial release of our CRM API allowed for basic searches across unified object types such as Accounts, Contacts, Leads, Opportunities, etc. In addition, we provided search capabilities that utilized the native formats such as SOQL for Salesforce and OData for Dynamics. However, researching how to form requests for each service is a tedious and time-consuming process. Therefore, we decided to release a new query language to interface with different CRM services through a common format.

Developers can now structure queries in the Kloudless Query Language (docs) to retrieve information from a CRM service via the Search endpoint. Some example KQL queries are included below and compared with their equivalent queries in upstream services.

Example 1: SugarCRM
/* KQL */
SELECT id, name, opportunity_type, probability, amount_usdollar,
  worst_case FROM Opportunities WHERE amount_usdollar >= 36000

/* SugarCRM equivalent */
    "filter": [
      {"amount_usdollar": {"$gte": 36000}}


Example 2: SugarCRM
/* KQL */
SELECT id, name, website FROM Accounts WHERE name LIKE 'We%'
  LIMIT 100 OFFSET 100

/* SugarCRM equivalent */
    "filter": [
      {"name": {"$starts": "We"}}


Example 3: Salesforce
/* KQL */
SELECT Id, Name, CreatedDate FROM Account WHERE Id IS NOT NULL
  AND CreatedDate > '2017-01-01T23:01:01+01:00'
  OR Name NOT LIKE '%b%'

/* SOQL equivalent */
SELECT Id, Name, CreatedDate FROM Account WHERE (
  ((Id != null) AND (CreatedDate > 2017-01-01T22:01:01Z))
  OR (NOT (Name LIKE '%b%')))

Future Plans

In the upcoming months, we plan on introducing additional connectors to the Kloudless CRM API as well as further improving features available. The next release of the Kloudless Query Language introduces support for unified properties to enable cross-account queries. Let us know what other improvements you’d like to see in our forums or on Twitter!

Kloudless Treehouse Adventure

Eight weeks previous, the Kloudless HQ team found themselves in a literal treehouse in Guerneville, California, for their annual retreat.

Image uploaded from iOS

The treehouse in question…

Image uploaded from iOS copy

…and the accompanying view!

The following is a vivified recount of the trip’s exciting festivities—and, for all of our readers, a fascinating peek behind the scenes into Kloud culture.

Day 1

Upon entering the treehouse, VP Engineering Tim, Kloudless’s resident man of culture, was greeted with one of his favorite things as a surprise birthday present from the entire team!


Tim and his beloved Johnnie Walker Blue Label Scotch Whisky

As soon as the whisky was safely delivered into Tim’s outstretched hands, beds were claimed, and the house was settled into, the team prepared to launch themselves into a bold exploration of the wild naturescapes and mysterious shorelines of Northern California…

First, our engineering team went to the beach to immerse themselves in the salty sea breeze and trademark chilly temperature characteristic of beaches of the area:


The elusive “outside,” the likes of which engineers are rarely able to witness.


Some of our engineering team looking fresh! I mean, just check out those shades on David “dthorman” Thorman, our Head of DevOps.


CTO Vinod, walking softly while also carrying a big stick.

Once the team satisfied themselves with their amount of exposure to the outside world, they settled back into their lodgings for the night, ending the day with food, drinks, and – naturally – a couple of intense games of Catan.

The centerpiece of the night? A delicious homemade paella cooked by Dthorman and Tim:

Image uploaded from iOS (1)

Fun fact: Tim is allergic to eggs, which is probably why he made paella instead of fried rice.

Soon enough, the Kloudless team found themselves falling into a groggy paella-and-whisky-induced slumber…  

Day 2

The next day of the retreat began with the team scarfing down a plentiful breakfast of fresh blueberries, croissants, pastries, scrambled and fried eggs (which Tim was unable to consume), bacon, and orange juice.


We have no shame in admitting that we got that box of croissants in the corner from Safeway.

The goal for this carboload-fest: absorb enough energy to sustain the body for the 4-hour-long kayaking trip down the Russian River that was scheduled in the itinerary for the day.


We weren’t sure why the water was green.


CEO Eliot (in center) with the rest of the team! Fun fact: when Eliot was in college, he was a star athlete on the Cal Dragonboat Team…which of course meant that everybody wanted to be paddled around in Eliot’s kayak.



Our lovely and chipper office manager Sophie and the effortlessly cool (though distinctly less chipper) Dthorman take a short break from rowing.


Vinod takes a break (and a selfie) while the rest of the team continues to row the kayak along the river. Fret not – no enmity arose in response to his break, even if his lack of help did make the boat harder to row.

For dinner, Sophie played head chef as she barked out orders to Tim (acting sous chef) to mince garlic and clean chicken legs. The finished result? An incredibly scrumptious and authentic dinner of fragrant Hainan chicken rice, with a comforting yet satisfying side of fishball and Napa cabbage soup:


If you can’t tell by now, food is a huge part of Kloud culture. You won’t find any Soylent bottles here.

Many thanks to Sophie’s dad for the recipe – the Kloudless team had a lot of fun making (and eating it)!

After dinner festivities included classic video games (Mario Kart and Super Smash Bros. Brawl), more board games (Codenames and more Catan), more drinks (Tim’s birthday present, as well as varieties of beer), and to top the night off, several rounds of poker.


Our software engineering intern Ellen “smashed” everyone at Smash, but Eliot cruised effortlessly into first place every single time at Mario Kart.


Software Engineering Intern Matthew Soh showing off his winnings for the night. (He cashed out at $8.50 with a high-roller buy in of $1 – at Kloudless, we endorse responsible gambling habits.)

All in all, a great retreat in the books for the Kloudless team! 

Join us!

Are you a board game or whiskey aficionado? Think you can beat Eliot at MarioKart, and Matthew at heads up poker?

Or do you simply want to enjoy some delicious paella, Hainan chicken rice, and ice cream with us? 

We’re hiring for engineering, marketing, and sales in our Berkeley and Taiwan offices! Find our job openings here.

Metrics and Monitoring at Kloudless

This post was written by our engineering intern, Matthew Soh.

Metrics and monitoring are important for any service. They provide critical information needed to detect and respond to incidents and issues. Kloudless deals with millions of requests, and keeping track of everything can be difficult. Kloudless has recently integrated a new metrics system to tackle this challenge.

Metrics from Kloudless can be sent to the new analytics platform for collection, analysis, and alerting. The analytics integration is available to all operators of Kloudless systems: both our own DevOps team administering our cloud version as well as operators of self-hosted Kloudless Enterprise appliances.


Let’s walk through a simple use case. Let’s say that your application uses the Kloudless Storage API and is failing to store files uploaded to your service. The potential issue could be anywhere in the stack. With the new metrics system, dashboards that display request metrics are easily accessible. Now you can quickly isolate the issue based on these metrics.


Chronograf dashboard with graphs of Kloudless metrics.

Here we have a simple dashboard. Core health checks to the Kloudless appliance appear to be fine. However, there is a sharp spike in the graph of Request Failures! This graph shows failures for outbound requests to upstream services. Hovering over the graph, we can look at the tags on each data series and see that there’s been a large increase in 500 errors to Box. This suggests that the issue is likely with the upstream service rather than the Kloudless API or the appliance itself. Knowing this allows us to narrow down which logs we need to look at to learn more about the error and take further steps to assess the root cause.

Request status is the mere tip of the iceberg when it comes to metrics provided by the Kloudless appliance. For a more detailed reference of available metrics, please refer to the Kloudless Enterprise Configuration guide.

How it works

The dashboard used above is built with Chronograf. It is one part of the metrics system deployed at Kloudless that uses the TICK stack, by InfluxData. TICK is comprised of Telegraf (collector), InfluxDB (datastore), Chronograf  (visualization), and Kapacitor (monitoring).


The TICK Stack. © 2017 InfluxData, Inc.

The metrics processing chain begins with Telegraf, the metrics collection daemon. Telegraf is designed to aggregate data from different sources and send them to various datastores. Sources include sysstat (a system information tool) and statsd (a common metrics daemon). The default datastore is InfluxDB, though others such as Graphite and CloudWatch can also be used. If required, the Telegraf output in the Kloudless appliance can be modified, enabling existing metrics collection or storage infrastructure to be used instead of InfluxDB.

The metrics then proceed to InfluxDB, which is designed for storing time-series data. This means that it has some neat features such as simple data retention policies and continuous queries. Data retention policies allow for time limits on metrics to expire old data. Continuous queries run at regular intervals on InfluxDB to summarise detailed data into broad overviews. For example, summations over counts of API requests are used to build daily summaries. Together, these features allow InfluxDB resource usage to be managed effectively.

Once the data is stored, Chronograf and Kapacitor work in tandem to help understand the collected metrics. Chronograf enables dashboard visualizations like the one described above to be built, while Kapacitor provides automatic monitoring so that there is no need to stare at the dashboard all day. Kapacitor’s monitoring and alerts are managed through TICKScripts which can be configured using either the Kapacitor command line client or Chronograf. We have provided sample TICKScripts which cover some common use cases.

Why we moved to the TICK stack

As the Kloudless Platform has grown, the volume and complexity of metrics data has grown with it. Previously, we were using with StatsD as a collector and Graphite to visualize metrics. This simple solution was easy to work with, but our needs have changed. Here are some of the unsupported scenarios we encountered:

  • StatsD doesn’t support tags. Tags are useful for providing context for the measurement, such as status or type of a request. Our workaround was to append tags onto the measurement name, similar to the tags used by DataDog’s DogStatsD. The downside of this approach was messy measurement names that were difficult to query.
  • Each StatsD measurement only has one value. It is sometimes useful to have a tuple of data grouped together in a single metric, or associate data such as application IDs to a metric. This isn’t possible with the StatsD+Graphite solution either.
  • The language used to query Graphite is limited to nesting of functions, and performing aggregations can be slow since the metrics are stored in flat files. This results in slow queries across multiple metrics and prevents easy use of complex operations.

These factors led us to look for a better alternative. InfluxDB seemed promising as its design was tailored for high volume time-series data and metrics collection. InfluxDB is built around measurements, which are in turn made up of many data points. A data point can have one or more values associated with it, and as many tags as needed. This addressed our first two issues right away and allowed for better querying.

For example, let’s try to determine which Kloudless applications were associated with failed API requests in the past day. We can do this with the following query:

select status, path, application_id from request_metrics_api_requests 
where time > now() - 24h and status=~/[^2][0-9]{2}/

In just one query, we’ve selected multiple values (status, path, application_id), filtered by tag values (status is not a 2XX code) and with a time frame limit (last 24 hours). This would have been messier and more tedious with our previous metrics system which did not allow associated metadata to be recorded. Additionally, InfluxDB’s time-series database design allows for tag-based queries to execute more efficiently since all tags are indexed.

Next Steps

We’ve been using the new metrics system at Kloudless to better understand, measure and monitor the performance of our hosted cloud platform. We’ve found that it has saved us time and effort when triaging issues, and we think you’ll find it useful as well. We’re excited to make this metrics system available to our enterprise customers using the Kloudless appliance. Customers using our cloud platform will also see analytics data for their Kloudless applications exposed in the developer portal in the upcoming months. As always, we would love to hear your thoughts and feedback on Twitter, comments below, or at hello@kloudless.com.

Calendar API: Availability Endpoint now… Available!

This post was written by our software engineer, Ryan Connor.

Finding an appropriate meeting time can take a lot of effort, even with just a few participants who have only a single calendar. Finding a meeting time that is sure to work for many participants with multiple calendars managed by several cloud services? Good luck!

Fortunately, Kloudless now helps you do just that. We are proud to announce that the Calendar Availability endpoint is online and ready to help your application find meeting times that work for any combination of user accounts and their calendars.

Using the Calendar Availability Endpoint

The Calendar Availability endpoint returns all available time windows among a set of calendars that match a specific meeting duration and desired time windows for the meeting. The endpoint also supports requests involving multiple calendars for multiple accounts.

To illustrate, imagine your app has 100 users who have two personal (Google) calendars and two work (Outlook) calendars for a total of four hundred calendars. One call to the Calendar Availability endpoint can retrieve available meeting times (if any) for all 100 users subject to all four of their calendars. Alternatively, you can retrieve available meeting times for just a subset of users and/or their calendars.

Kloudless seamlessly integrates with Google and Outlook Calendar behind the scenes—all you have to provide are Account IDs, Calendar IDs, a meeting duration, and time constraints. Let’s take a look at exactly how to do that.

Formatting the Request and Parsing the Response

Here are the details for the request and response format of the Calendar Availability endpoint based on our docs. You can scroll down further to see concrete examples.

Request Format

URL: https://api.kloudless.com/v1/accounts/{account_id,account_id,…}/cal/availability

Method: POST


  • Authorization: Bearer [Token]
  • Content-Type: application/json


  • calendars: List of Calendar IDs. Uses the default calendar if empty. (Optional)
  • meeting_duration: ISO 8601 format for duration. (Required)
  • constraints: A dictionary of constraint keys and values. (Required)
    • time_windows: List of desired time slots with the following values.
      • start: ISO 8601 datetime format
      • end: ISO 8601 datetime format

Response Format


  • Content-Type: application/json


  • time_windows: List of desired time slots with the following values.
    • start: ISO 8601 datetime format
    • end: ISO 8601 datetime format

The start and end times of each time_window in the response bookend (inclusively) the time periods in which all of the accounts are available given the constraints. Note that the times are returned in the GMT time zone, so you may want to convert to the time zone of your choice.

Concretely, if the response includes the “2 – 5 PM” time window and you wanted a 30-minute meeting, you can safely schedule the meeting to start and end at any time within 2 – 5 PM, inclusive (e.g., 2 – 2:30, 2:01-2:31, …, 4:29-4:59, 4:30-5:00).

Example Usage

Single account, two calendars specified

curl -H 'Authorization: Bearer [TOKEN]' \
    -H 'Content-Type: application/json' \
    -XPOST -d '{
        "calendars": ["ra5werWRsZXQzLcRik5BudGltb6RwwUNnbWFpbC5jb21=",
        "meeting_duration": "PT1H",
        "constraints": {
            "time_windows": [{
                "start": "2017-05-20T08:00:00+07:00",
                "end": "2017-05-20T12:00:00+07:00"
                "start": "2017-05-21T08:00:00+07:00",
                "end": "2017-05-21T12:00:00+07:00"
    }' \


  "time_windows": [
      "start": "2017-05-20T02:00:00Z",
      "end": "2017-05-20T04:00:00Z"
      "start": "2017-05-21T03:00:00Z",
      "end": "2017-05-21T04:00:00Z"

In this case, the requestor wants to find an appropriate meeting time for a one hour meeting during the 8 AM – 12 PM GMT+7 time window on either May 20, 2017 or May 21, 2017. The requestor is requiring one account as a participant in the meeting and further specifying two calendars from that account.

Note that the primary calendar within an account is automatically considered if no calendars are provided. But, if any calendars are provided, the primary calendar must be included to be considered. For example, if the primary calendar ID in this case is neither ra5werWRsZXQzLcRik5BudGltb6RwwUNnbWFpbC5jb21= nor fa2xvdWRsZXNzLnRlc3QudGltb3RoeUBnbWFpbC5jb20=, the primary calendar would be ignored when retrieving availability. To also consider the primary calendar, the requestor should provide it as a third calendar in the given calendar list.

The response indicates that any one-hour slot between either 9 AM – 11 AM GMT+7 on May 20, 2017 or 10 AM – 11 AM on May 21, 2017 works for the meeting. Of course, the desired meeting length is exactly as long as the boundaries of the second available time window, so there is only one slot that works on that date.

Multiple accounts, no calendars specified

curl -H 'Authorization: Bearer [TOKEN]' \
    -H 'Content-Type: application/json' \
    -XPOST -d '{
        "meeting_duration": "PT45M",
        "constraints": {
            "time_windows": [{
                "start": "2017-06-15T08:00:00-03:00",
                "end": "2017-06-15T17:00:00-03:00"
    }' \
  "time_windows": [
      "start": "2017-06-15T05:00:00Z",
      "end": "2017-06-15T06:30:00Z"
      "start": "2017-06-15T07:45:00Z",
      "end": "2017-06-15T10:00:00Z"
      "start": "2017-06-15T12:30:00Z", 
      "end": "2017-06-15T14:00:00Z"

Here, the requestor wants to find a time for a 45-minute meeting during the 8 AM – 5 PM GMT-3 time window on June 15, 2017. The requestor is requiring three accounts as participants in the meeting, and since no calendars are specified, is searching for availability based solely on each account’s primary calendar.

Note that you can pass time window start and end times in any time zone (and, in fact, differing time zones within the same request). Furthermore, the calendars to be searched on need not be in the same time zone as the constraints. The important point is that the times are passed as ISO 8601-formatted strings—Kloudless will handle the rest.

The response indicates that any 45-minute slot between 9 AM – 10:30 AM GMT-3, 0:45 AM – 1 PM GMT-3, or 3:30 – 5 PM GMT-3 on June 15, 2017 works for the meeting.

Looking to the Future

The Calendar Availability endpoint provides a simple way to coordinate meetings among multiple calendars and calendar accounts. Going forward, we will monitor usage of the endpoint and listen to your feedback to determine the best way to expand the endpoint’s functionality and flexibility. We already have some great suggestions for making the endpoint more customizable, including:

  • The start_times constraint. If the requestor provides start_times in addition to (or in lieu of) time_windows, the endpoint would return the subset of start_times that work as the start time for the meeting. Essentially, this lets the requestor ask “Which of these exact start times are OK for the meeting?”
  • The ​min_attend_percent constraint. Right now, a time window will only be included in the response if every given account is available in that window—that is, if one relevant calendar in one account is not available during a window, that account is not available and so that window is not returned. With the min_attend_percent constraint, a time window would be included if at least the specified percentage of accounts are available during that window.e
  • The work_or_personal constraint. The requestor could provide a work_or_personal argument to limit the times considered for work or personal hours. This could be especially useful when trying to coordinate across widely varying time zones. For example, a colleague across the world may technically be “available” for a meeting at the time the requestor desires, but that’s of little use if it’s 2 AM for that colleague!

Wrapping Up

We’re excited to rollout the Calendar Availability endpoint to the developers on our platform. What do you like about the endpoint right now, what questions do you have about it, and what do you wish it would do in the future? We would love to hear your thoughts and feedback on Twitter, our developer forums, comments below, or at hello@kloudless.com.


Calendar API: Activity Stream

This post is by our engineering intern, Ellen Luo.

Receive real-time updates for Google and Outlook Calendar with the new Events endpoint for the Calendar API!

The Events endpoint allows your application to easily keep track of calendar items that have been added, modified or deleted in your primary calendar. This Events endpoint is part of our broader Events API to monitor activity in connected accounts, and should not be confused with the Calendar Events API that enables access to calendar appointments.

Retrieving activity data with the Events API

To start using the Events API, check the box beside “Collect Events” in the App Details page of the Developer Portal. Kloudless will automatically begin to collect activity data for any newly connected Google or Outlook Calendar accounts. Google Calendar allows real-time updates of changes to users’ accounts so event data is updated in at most 1.5 minutes. For Outlook Calendar, Kloudless queries the service repeatedly to find changes. This means that event data will be updated on average every 15 minutes in our cloud and 1-5 minutes in private installs (configurable).

Requesting activity data

To return a list of all recent events collected, make a request to https://api.kloudless.com/v1/accounts/{account_id}/events/ with your connected account’s ID and Bearer token (or application API Key). A sample request is shown below:

curl -X GET -H 'Accept: application/json' \
    -H 'Authorization: Bearer {token}' \

Parsing the response

The response will contain a list objects of all the events. Each object has a type, which indicates the type of change. Right now, the only events that are detected are “add” (object created), “update” (existing object modified), and “delete” (object deleted). Each event object will have the updated metadata of the object modified. More information about the Events API and sample responses can be found in the Kloudless docs here.


In the future, we will add support for retrieving activity data for calendars other than the account’s primary calendar. We will also be working towards including more specific event types and expanding support to more calendaring APIs. We’re excited to make the Calendar API Activity Stream endpoint available to all developers on our platform and would love to hear any feedback or suggestions on Twitter, the comments below, or at hello@kloudless.com.

Office 365 PowerShell queries via REST: Maximizing the Kloudless Pass-through API

In our previous post, we announced the availability of the Kloudless Pass-through API. The Pass-through API enables your application to make API requests directly to third-party services, while still using Kloudless’s unified APIs. In this blog post, we’ll discuss how to access the Office 365 PowerShell via the Kloudless REST API to perform administrative tasks in Office 365.

Building with our easy-to-use REST API offers many benefits. We handle the complexities of integrating with each service behind the scenes so you  don’t have to. This speeds up your integration time and decreases future maintenance. We’ve extended the same principle to our Pass-through API by introducing special capabilities such as enabling PowerShell queries for Office 365 admin accounts.

Office 365 PowerShell provides several remote management commands that can be used to administer your Office 365 tenant, similar to how you would via the Office 365 admin center web application. For example, a broad set of security and compliance features can be accessed via theSecurity & Compliance Center cmdlets by connecting to Office 365 using remote PowerShell. Normally, you would need access to a PowerShell prompt in order to access this functionality. The Kloudless REST API handles the heavy-lifting and enables you to access this functionality via our REST API.

Invoking Office 365 Security Center cmdlets via the Kloudless Pass-through API

To begin, connect a SharePoint Online admin account to your Kloudless application. The easiest way to do this is by logging into your Kloudless account and then navigating to the Interactive Docs. Click the “Add Account” button and then click on SharePoint Online under the “Admin accounts” section towards the bottom of the pop-up that opens.


Once you’ve connected your account, you will receive a Kloudless Account ID that can be used for API requests to the Kloudless API. You are now ready to make Pass-through API requests to this admin Office 365 account!

While SharePoint Online REST API requests can be performed without any additional configuration, the PowerShell queries described in this blog post are available in Kloudless Enterprise and also require special permission to access by Kloudless Enterprise developers. Please contact us at support@kloudless.com to learn how to enable this capability for your Kloudless Enterprise instances.

Request Format

The format of PowerShell pass-through API requests is as follows:

URL: https://api.kloudless.com/v1/accounts/{account_id}/raw
  • {account_id} is the Kloudless account ID of the SharePoint Online admin account connected.
Headers (described in the Pass-through API docs):
  • X-Kloudless-Raw-URI: http://powershell/ This special value indicates the request should be translated to a PowerShell query.
  • X-Kloudless-Raw-Method: POST
  • Authorization: Bearer {account_bearer_token} OR Authorization: APIKey {application_api_key} See our Authentication Docs for more information on authorizing API requests.
JSON data in the format below:

      "category": "o365-security",
      "command": {cmdlet_name},
      "options": {
        ... option name: value mappings if required ...
At the current time, only Office 365 Security and Compliance Center cmdlets ("category": "o365-security") and Exchange Online cmdlets ("category": "exchange") are available via the Kloudless API. If you would like access to other remote PowerShell cmdlets, please contact us at support@kloudless.com.

Examples of Requests

An example of a curl request with the format described above would be:

curl -H "Authorization: APIKey {api_key}" \
     -H "X-Kloudless-Raw-URI: http://powershell/ \
    https://api.kloudless.com/v1/accounts/{account_id}/raw \
    --data '{body}'

Please replace the {api_key}, {account_id} and {body} values with your API Key, connected account’s ID and JSON data for PowerShell respectively.

Here are some examples of Body data to use in {body} for specific cmdlets:

Obtaining a list of compliance cases.

  "category": "o365-security",
  "command": "Get-ComplianceCase"
An example of a curl request for this would be:

curl -H "Authorization: APIKey 123ABC" \
     -H "X-Kloudless-Raw-URI: http://powershell/" \
     https://api.kloudless.com/v1/accounts/123/raw \
     --data '{"category": "o365-security", "command": "Get-ComplianceCase"}'
Create a new compliance case.

  "category": "o365-security",
  "command": "New-ComplianceCase",
  "options": {
    "Name": "test new case 2",
    "Description": "This case is created via curl"
An example curl request would be identical to the one used for Get-ComplianceCase but with the new value above for --data instead.
Create a new hold policy for a case.

  "category": "o365-security",
  "command": "New-CaseHoldPolicy",
  "options": {
    "Case": "3b4de8d5-13cb-4291-bdd0-b6e2bb82a08e",
    "Name": "New Hold",
    "SharePointLocation": "https://kloudless.sharepoint.com/test subsite/"
where "3b4de8d5-13cb-4291-bdd0-b6e2bb82a08e" is the Identity GUID of the Case to add the legal hold policy to. This corresponds to the Locations section of a Hold when editing a Case’s Holds at https://protection.office.com/#/ediscovery.
Create a rule to add to a hold policy.

  "category": "o365-security",
  "command": "New-CaseHoldRule",
  "options": {
    "Policy": "New Hold",
    "Name": "New Rule",
    "ContentMatchQuery": "SSN"
where "New Hold" is the name of the hold I created previously. This corresponds to the Conditions section of a Hold.

Similarly, other policies such as retention policies can also be created, and existing objects can be deleted:

Create a new preservation policy.

  "category": "o365-security",
  "command": "New-RetentionCompliancePolicy",
  "options": {
    "Name": "Test new policy",
    "SharePointLocation": "https://kloudless.sharepoint.com/"
This creates a policy but has not yet created a rule to add to it, which can be done with .
Deleting a compliance case.

  "category": "o365-security",
  "command": "Remove-ComplianceCase",
  "options": {
    "Identity": "94b99324-5574-4220-b081-1b689cb386af",
    "Confirm:$false": null
where "94b99324-5574-4220-b081-1b689cb386af" is the Identity GUID of the Case to remove.

Future capabilities of the Pass-through API

The Pass-through API provides provides a powerful new way to access third-party features via Kloudless as shown above. We’re excited to make this available to all developers on our platform and would love to hear any feedback or suggestions in our developer forum.