Kloudless Walkthrough: Building with our Open-Source Authenticator JS Library

This post was written by our engineer Chris Prochnow.

In this walkthrough, we’ll use the Kloudless open-source Authenticator JS library that prompts the user to grant access to a cloud service. We will look at the set-up of a basic web server, and explore how to use the Authenticator to set up a client-side form with all the data needed to send to the server-side for API requests.

We’ve written Authenticator.JS documentation that we will reference through the article. You can find it on GitHub here.

Getting started

Here is a form with all the fields needed to create a basic text file, which will eventually be stored in the Google Drive account that the user has been granted permission to access.

html


<form action="/post" method="post">
  <span>
    <input type="text" name="fileName" placeholder="File Name">
  </span>
  <span>
    <textarea name="fileBody&quot; rows="7">File text here.</textarea>
  </span>
  <span>
    <button type="submit">Create and upload file</button>
  </span>
  <input type="hidden" name="access_token" id="access_token" value="">
  <input type="hidden" name="account_id" id="account_id" value="">
</form>

Our web server is based on Node and Express. We will use the submitted form data to create a file, verify the authentication token, construct an API call, and run some cleanup once the action is complete. If you look at the code above, there are two inputs with the type=“hidden” that do not yet contain values. The values account_id and access_token must be supplied to us in order to access the storage service. Here is where the Authenticator JS library comes in.

First, we must obtain the correct values. To do so, we’ll need to embed the Authenticator in the client-side page using this script tag, which is documented in the Authenticator’s README here.

html

form2

This exposes a global Kloudless object, which contains the  Kloudless.authenticator method. This method is what we’ll use to trigger an authentication pop-up for the end-user:

Kloudless.authenticator(element, params, callback)

Before that can happen, we need to set up some logic to handle the end-user’s input: 

  • element is the triggering action. For this example, I’ve used jQuery to set the triggering element.
  • We also need to pass the configuration to the Kloudless.authenticator method to suit our application needs. These are required parameters that the end-user does not need to be aware of, but that do need to be passed in to the authentication.
  • At the minimum, you must pass the Kloudless application ID to identify the Kloudless app. This is provided as the client_id.

In our case, we only need to authenticate one service, Google Drive, so we will also be setting the scope to 'gdrive' in our configuration. The config will look like this:

  const config = {
      'client_id': ‘[YOUR CLIENT ID HERE]’,
      scope: 'gdrive',
  };

This gives a great setup for the authenticator.js!

Let’s see how what we have so far could be laid out:

$(document).ready(() => {
  const config = {
      'client_id': ‘[YOUR CLIENT ID HERE]’,
      ‘scope’: 'gdrive',
  };
  let auth = Kloudless.authenticator(press, config, callback);
});

Pairing data with input sources

The next pieces we need to include will be adding additional JS to set our received data to the correct inputs in our form, and setting a callback function that will determine what we do with the response data. The JS setup is pretty quick and involves defining some variables required using jQuery.

const $press = $('#submitAuth');
const $account_id = $('#account_id');
const $access_token = $('#access_token');
const $uploadButton = $('#uploadButton');

Callback function

Now for the callback function setup. We’ll need to handle errors, as well as decide what we’ll do with our successful data. Using the variables we just set up, we can reset the input values with the returned information:

let callback = function (result) {
   if (result.error) {
     console.log('An error occurred:', result.error);
     return;
   }   

    $account_id.val(result.account.id);
   $access_token.val(result.access_token);
   $uploadButton.prop('disabled',false);
 };

Our authenticator.js should now look something like this:

$(document).ready(() => {

 const config = {
     'client_id': 'NyRVsMqKtYS7g9t40WuSqhJCAxi4ANZ3p6MOsKfCFjXX_xcb',
     'scope': 'gdrive',
 };

 const $press = $('#submitAuth');
 const $account_id = $('#account_id');
 const $access_token = $('#access_token');
 const $uploadButton = $('#uploadButton');

 let callback = function (result) {
    if (result.error) {
      console.log('An error occurred:', result.error);
      return;
    }

    $account_id.val(result.account.id);
    $access_token.val(result.access_token);
    $uploadButton.prop('disabled',false);

  };

  let auth = Kloudless.authenticator($press, config, callback);

});

With that, we now have all the data needed to send to our server-side code to make an API call. This initial set-up uses an OAuth Kloudless key, which will be sufficient for basic development needs, but will be subject to rate-limit restrictions.


Setting up your own OAuth credentials

The next part of this blog will discuss how you can set up your own OAuth credentials. We’ll continue to use Google Drive as our example. 

Setting up these credentials provides several benefits: it allows you to manage your own tokens, implement a better branding experience, and exempt you from sharing the rate limits/quotas that Kloudless default keys are subject to. So let’s dive in!

To start off, navigate to the OAuth Credentials page in the developer portal for your app. You can find this page under Security -> Credentials in the developer portal. Be sure to select the correct app from the dropdown menu in the top left corner of the page.

You should see a list of cloud storage services that looks something like this:

credentials1.png

In our example, we are looking for the listing for Google Drive. When you find that entry, you will see that Google Drive is by default set to the Default Kloudless keys.

If you have existing accounts connected to the Default key, you will see a message denoting the number of accounts using that key, and an option to delete these accounts bordered in red, with some additional information.

credentials2.png

We do not need these accounts to continue to have access, and would like to have them re-authenticate through our app in the future. In this case I will check the box to have the existing accounts deleted.

Please note: this deletion does not affect the contents of your account or delete the account, but they do remove access to the account the keys are associated with.

Once you click ‘Save’, the first thing you’ll see is several new fields with instructions and helpful information specific to OAuth tokens and Google.

credential3.png

Keep this browser tab open—we’ll be coming back to it in a bit.

We’re now ready to head over to the Google Developers Console!

If you haven’t yet, create a new Google project. This is where you’ll manage apps and services that request access to your users’ Google accounts, and generate a Client ID/Consumer Key and our Client Secret/Consumer Secret.

Once that is done, there are a few steps to follow to get the information from the Google project. It should look something like the image below once you have created the project, with your project name in place of ‘kloudless test auth’:

credentials4.png

Our next step is to create the credentials needed for the Kloudless app. Click on the ‘Credentials’ tab to look at our options for creating credentials. In this case, we’re creating an OAuth Client ID, so select that option and then select the correct application type. In this case, I selected ‘Web application’.

All right! We now have a Client ID/Consumer Key and our Client Secret/Consumer Secret.

We’ve got something else to edit in the Google Developers Console: the callback URL or Redirect URL. Refer back to the Kloudless Credentials page and grab the link supplied there. Copy and paste that into the field under ‘authorized redirect URIs’, and click save.

credentials5.png

Note: keep in mind that the Client secret is just that—a SECRET. Be mindful of where you save it, and do not leave it exposed in your documentation.

Finally, head back to the Kloudless Credentials page. Enter the Client ID and Client secret into their respective fields, and save the app.

That’s it—you’ve successfully moved your app from using the default Kloudless OAuth keys to your own OAuth keys! 😀🎉


Conclusion

We hope that you found our introduction to the Kloudless Authenticator JS library useful. As we grow, we’ll be expanding our open-source initiatives—and the accompanying guides, walkthroughs, and other materials—so stay tuned for more posts like this one.

Want to build on these initiatives? Come join us—we’re hiring across a ton of different functions in our Berkeley, CA and Taipei, TW offices!

Securing enterprise data across cloud services

We’re right around the corner from this year’s RSA Conference, held in San Francisco from April 16-20! As the conference approaches, it’s the perfect time for enterprises to evaluate their security priorities for the coming year.

What’s new in cyber security?

Interest in cyber security solutions has risen dramatically this decade. We know that the average enterprise is increasingly adopting online cloud-based SaaS tools for use in a variety of situations, from workflows and productivity to software development. But as data moves from traditional on-premise content repositories to the cloud, data becomes fragmented across a multitude of ecosystems. As a result, new technologies are emerging to protect data outside the safety of a corporate firewall.

Source: Stacy Jones from Fortune graphs Wade Baker’s analysis of RSA data.

Given the sensitive nature of security in the enterprise, it’s imperative for an enterprise’s security initiatives to align and adapt to a continuously evolving threat landscape. Yet it’s no ordinary feat to uniformly and reliably monitor activity on every disparate cloud service used by an organization.

Announcing new Kloudless Security APIs

When we launched the first Kloudless unified API a few years ago, we didn’t expect the variety of data security solutions that we would enable. Several of our users, customers, and partners are RSAC sponsors, where they showcase security solutions connecting to business applications like Office 365 and G Suite via Kloudless.

As it turns out, our unified APIs offer strong benefits designed for apps offering security solutions for cloud services, including, but not limited to:

  • Capturing information on all organization data in the cloud service, and who can access it;
  • Monitoring any and all activity occurring organization-wide;
  • Remediating threats using the Kloudless unified APIs for Storage, CRM, ITSM, and more;
  • Running Kloudless Docker containers on-premises to meet and exceed custom security and compliance requirements.

We’re dedicating more time this quarter to improve our security solutions with robust and relevant features that our users have been asking for. Today, we’re excited to announce two new versions of API endpoints targeted specifically for security integrations:

  • Unified Audit Events
    • Kloudless makes a stream of event data available in a consistent format. This includes information on any data being modified as well as behavioral events like logins, sharing, user/group creation, etc.  

  • Unified Permissions
    • Access controls are vital to any security company, and these controls can be extremely complex. The Kloudless v2 Permissions API provides granular information on the various access levels available on objects.

See you at RSA!

We’re looking to launch additional data security solutions in the upcoming months to complement the suite of v2 endpoints we released today. At the RSA Conference, we’ll be learning more about current trends, so we can include the most relevant features in our solutions.

Interested in meeting up at RSA? Drop us a line at hello@kloudless.com!   

Kloudless raises $6M to ease the software integration process

Website graphic

We’re thrilled to announce that we’ve raised $6M in Series A funding in a round led by Aspect Ventures, with participation by Bow Capital, Alibaba Taiwan Entrepreneur Fund, Heavybit, and Ajay Shah of Silver Lake Capital!

With its one-to-many approach, Kloudless Unified APIs free developers and product teams from building tedious and expensive one-off integrations. Try it out here.

At Kloudless, we have a firsthand understanding of how frustrating it can be to build the integrations we need. In our increasingly interconnected world, new SaaS services launch daily, and each one offers a unique API with its own documentation and features. It’s little wonder that API usage has exploded in the last few years.

In 2015, our team set out to change the way integrations were built by creating a developer-first solution to the problem—and in the process, became the first integrations platform to offer a solution tailored to the unique needs of developer and product teams. Our Unified APIs eliminate the redundancies of building integrations and now free thousands of developers to focus on their core product.

Since then, we’ve:

  • Grown from our small Berkeley-based team of 5 to 20 (and launched our first-ever international office in Taipei, Taiwan);
  • Welcomed Blackboard Inc., PlanGrid, ShieldX, OneIdentity, and other great companies to our platform;
  • Built up a strong community of over 15,000 developers from 4,000 different organizations.

We’re immensely proud of how far we’ve come and are equally excited to start the next chapter of the Kloudless journey!

What’s next?

Our new round of funding will help Kloudless to continually grow the success of our users, our platform, and our team.

Some of the technical improvements we’ve already planned include:

  • Boilerplate API integrations for any SaaS service’s HTTP API, with the help of automation
  • A new framework for workflow automation built on top of Kloudless’ extensive API platform
  • Expanding API unification and cross-connector capabilities like KQL
  • Additional production-ready UI tools to solve workflow use cases for companies in any industry

Of course, there’s more to come—so stay tuned!

Built by developers, for developers

Our new round of funding also makes it possible for us to make a concerted investment in initiatives that ensure the vibrancy and connectivity of the developer community.

As our users grow, we’ll create high-quality documentation, strengthen our SDK offerings, and cultivate responsive community forums to ensure that the experience of building with Kloudless continues to be efficient (and, dare we say, enjoyable!). Our efforts will focus on the most common use cases that our users encounter when integrating with other services.

Conclusion

We’re excited to tackle our road map and delight our users with new Kloudless platform expansions and features. To our existing Kloudless users, thank you for your support all these years – we wouldn’t have gotten here without you.

Haven’t tried Kloudless before? Sign up for a free account here to get started today!

We’re also hiring for a bunch of different functions, from Marketing to Sales to Engineering, in our Berkeley and our Taiwan offices. Check out our listings here!

About Kloudless

Kloudless’s unified APIs enable developers to code once to connect their applications to several software services instead of integrating with each API individually. Businesses use Kloudless’s unified API products to connect to many kinds of enterprise applications like Salesforce, SharePoint, Box, Dropbox, G Suite, Slack, ServiceNow and more.

Kloudless is trusted by more than 15,000 developers in over 4,000 organizations. To learn more, visit https://kloudless.com.

Introducing the Kloudless Query Language and new CRM connectors

Kloudless initially launched the unified CRM API with support for Salesforce and Microsoft Dynamics, enabling developers to connect apps with two large open SaaS CRM platforms. The initial release provided core features such as CRUD operations for all objects, near real-time activity monitoring, and making batch requests.

We often think of Salesforce, Microsoft, and SAP as the largest players in the space, but Gartner reports show that they only account for 35% of the CRM market share.

We are excited to announce support for a number of other CRM services in our unified CRM API, including Oracle Sales Cloud, SugarCRM, Hubspot CRM, Zoho CRM, and Pipeliner CRM. Existing customers using the unified CRM API can now offer these integrations to their users with minimal additional effort.

Here is a current breakdown of feature availability of the CRM API across the CRM services we support:

Salesforce Dynamics Oracle Pipeliner Zoho SugarCRM Hubspot
OAuth
CRUD
Search        ⏱
Events ✓*

: Supported            ✓*: Partially supported            ⏱: TBD

Kloudless Query Language (KQL)

We are also excited to announce a common query language to access object data in any supported CRM service.

The initial release of our CRM API allowed for basic searches across unified object types such as Accounts, Contacts, Leads, Opportunities, etc. In addition, we provided search capabilities that utilized the native formats such as SOQL for Salesforce and OData for Dynamics. However, researching how to form requests for each service is a tedious and time-consuming process. Therefore, we decided to release a new query language to interface with different CRM services through a common format.

Developers can now structure queries in the Kloudless Query Language (docs) to retrieve information from a CRM service via the Search endpoint. Some example KQL queries are included below and compared with their equivalent queries in upstream services.

Example 1: SugarCRM
/* KQL */
SELECT id, name, opportunity_type, probability, amount_usdollar,
  worst_case FROM Opportunities WHERE amount_usdollar >= 36000

/* SugarCRM equivalent */
https://<site_url>/rest/v10/Opportunities/filter?fields=id,name,
opportunity_type,probability,amount_usdollar,worst_case
  {
    "filter": [
      {"amount_usdollar": {"$gte": 36000}}
    ]
  }

 

Example 2: SugarCRM
/* KQL */
SELECT id, name, website FROM Accounts WHERE name LIKE 'We%'
  LIMIT 100 OFFSET 100

/* SugarCRM equivalent */
http://<site_url>/rest/v10/Opportunities/filter?fields=id,name,
website&offset=100&max_num=100
  {
    "filter": [
      {"name": {"$starts": "We"}}
    ]
  }

 

Example 3: Salesforce
/* KQL */
SELECT Id, Name, CreatedDate FROM Account WHERE Id IS NOT NULL
  AND CreatedDate > '2017-01-01T23:01:01+01:00'
  OR Name NOT LIKE '%b%'

/* SOQL equivalent */
SELECT Id, Name, CreatedDate FROM Account WHERE (
  ((Id != null) AND (CreatedDate > 2017-01-01T22:01:01Z))
  OR (NOT (Name LIKE '%b%')))

Future Plans

In the upcoming months, we plan on introducing additional connectors to the Kloudless CRM API as well as further improving features available. The next release of the Kloudless Query Language introduces support for unified properties to enable cross-account queries. Let us know what other improvements you’d like to see in our forums or on Twitter!

Kloudless Treehouse Adventure

Eight weeks previous, the Kloudless HQ team found themselves in a literal treehouse in Guerneville, California, for their annual retreat.

Image uploaded from iOS

The treehouse in question…

Image uploaded from iOS copy

…and the accompanying view!

The following is a vivified recount of the trip’s exciting festivities—and, for all of our readers, a fascinating peek behind the scenes into Kloud culture.

Day 1

Upon entering the treehouse, VP Engineering Tim, Kloudless’s resident man of culture, was greeted with one of his favorite things as a surprise birthday present from the entire team!

IMG_6848

Tim and his beloved Johnnie Walker Blue Label Scotch Whisky

As soon as the whisky was safely delivered into Tim’s outstretched hands, beds were claimed, and the house was settled into, the team prepared to launch themselves into a bold exploration of the wild naturescapes and mysterious shorelines of Northern California…

First, our engineering team went to the beach to immerse themselves in the salty sea breeze and trademark chilly temperature characteristic of beaches of the area:

IMG_6830

The elusive “outside,” the likes of which engineers are rarely able to witness.

20170815_152259

Some of our engineering team looking fresh! I mean, just check out those shades on David “dthorman” Thorman, our Head of DevOps.

IMG_6841

CTO Vinod, walking softly while also carrying a big stick.

Once the team satisfied themselves with their amount of exposure to the outside world, they settled back into their lodgings for the night, ending the day with food, drinks, and – naturally – a couple of intense games of Catan.

The centerpiece of the night? A delicious homemade paella cooked by Dthorman and Tim:

Image uploaded from iOS (1)

Fun fact: Tim is allergic to eggs, which is probably why he made paella instead of fried rice.

Soon enough, the Kloudless team found themselves falling into a groggy paella-and-whisky-induced slumber…  


Day 2

The next day of the retreat began with the team scarfing down a plentiful breakfast of fresh blueberries, croissants, pastries, scrambled and fried eggs (which Tim was unable to consume), bacon, and orange juice.

DCIM100GOPROGOPR2143.JPG

We have no shame in admitting that we got that box of croissants in the corner from Safeway.

The goal for this carboload-fest: absorb enough energy to sustain the body for the 4-hour-long kayaking trip down the Russian River that was scheduled in the itinerary for the day.

DCIM100GOPROGOPR2145.JPG

We weren’t sure why the water was green.

DCIM100GOPROGOPR2148.JPG

CEO Eliot (in center) with the rest of the team! Fun fact: when Eliot was in college, he was a star athlete on the Cal Dragonboat Team…which of course meant that everybody wanted to be paddled around in Eliot’s kayak.

 

20170816_140622

Our lovely and chipper office manager Sophie and the effortlessly cool (though distinctly less chipper) Dthorman take a short break from rowing.

20170816_140630.png

Vinod takes a break (and a selfie) while the rest of the team continues to row the kayak along the river. Fret not – no enmity arose in response to his break, even if his lack of help did make the boat harder to row.

For dinner, Sophie played head chef as she barked out orders to Tim (acting sous chef) to mince garlic and clean chicken legs. The finished result? An incredibly scrumptious and authentic dinner of fragrant Hainan chicken rice, with a comforting yet satisfying side of fishball and Napa cabbage soup:

20170816_183405

If you can’t tell by now, food is a huge part of Kloud culture. You won’t find any Soylent bottles here.

Many thanks to Sophie’s dad for the recipe – the Kloudless team had a lot of fun making (and eating it)!

After dinner festivities included classic video games (Mario Kart and Super Smash Bros. Brawl), more board games (Codenames and more Catan), more drinks (Tim’s birthday present, as well as varieties of beer), and to top the night off, several rounds of poker.

20170816_204649

Our software engineering intern Ellen “smashed” everyone at Smash, but Eliot cruised effortlessly into first place every single time at Mario Kart.

20170817_010850

Software Engineering Intern Matthew Soh showing off his winnings for the night. (He cashed out at $8.50 with a high-roller buy in of $1 – at Kloudless, we endorse responsible gambling habits.)

All in all, a great retreat in the books for the Kloudless team! 


Join us!

Are you a board game or whiskey aficionado? Think you can beat Eliot at MarioKart, and Matthew at heads up poker?

Or do you simply want to enjoy some delicious paella, Hainan chicken rice, and ice cream with us? 

We’re hiring for engineering, marketing, and sales in our Berkeley and Taiwan offices! Find our job openings here.

Metrics and Monitoring at Kloudless

This post was written by our engineering intern, Matthew Soh.

Metrics and monitoring are important for any service. They provide critical information needed to detect and respond to incidents and issues. Kloudless deals with millions of requests, and keeping track of everything can be difficult. Kloudless has recently integrated a new metrics system to tackle this challenge.

Metrics from Kloudless can be sent to the new analytics platform for collection, analysis, and alerting. The analytics integration is available to all operators of Kloudless systems: both our own DevOps team administering our cloud version as well as operators of self-hosted Kloudless Enterprise appliances.

Usage

Let’s walk through a simple use case. Let’s say that your application uses the Kloudless Storage API and is failing to store files uploaded to your service. The potential issue could be anywhere in the stack. With the new metrics system, dashboards that display request metrics are easily accessible. Now you can quickly isolate the issue based on these metrics.

blog-post-dashboard

Chronograf dashboard with graphs of Kloudless metrics.

Here we have a simple dashboard. Core health checks to the Kloudless appliance appear to be fine. However, there is a sharp spike in the graph of Request Failures! This graph shows failures for outbound requests to upstream services. Hovering over the graph, we can look at the tags on each data series and see that there’s been a large increase in 500 errors to Box. This suggests that the issue is likely with the upstream service rather than the Kloudless API or the appliance itself. Knowing this allows us to narrow down which logs we need to look at to learn more about the error and take further steps to assess the root cause.

Request status is the mere tip of the iceberg when it comes to metrics provided by the Kloudless appliance. For a more detailed reference of available metrics, please refer to the Kloudless Enterprise Configuration guide.

How it works

The dashboard used above is built with Chronograf. It is one part of the metrics system deployed at Kloudless that uses the TICK stack, by InfluxData. TICK is comprised of Telegraf (collector), InfluxDB (datastore), Chronograf  (visualization), and Kapacitor (monitoring).

blog-post-influx

The TICK Stack. © 2017 InfluxData, Inc.

The metrics processing chain begins with Telegraf, the metrics collection daemon. Telegraf is designed to aggregate data from different sources and send them to various datastores. Sources include sysstat (a system information tool) and statsd (a common metrics daemon). The default datastore is InfluxDB, though others such as Graphite and CloudWatch can also be used. If required, the Telegraf output in the Kloudless appliance can be modified, enabling existing metrics collection or storage infrastructure to be used instead of InfluxDB.

The metrics then proceed to InfluxDB, which is designed for storing time-series data. This means that it has some neat features such as simple data retention policies and continuous queries. Data retention policies allow for time limits on metrics to expire old data. Continuous queries run at regular intervals on InfluxDB to summarise detailed data into broad overviews. For example, summations over counts of API requests are used to build daily summaries. Together, these features allow InfluxDB resource usage to be managed effectively.

Once the data is stored, Chronograf and Kapacitor work in tandem to help understand the collected metrics. Chronograf enables dashboard visualizations like the one described above to be built, while Kapacitor provides automatic monitoring so that there is no need to stare at the dashboard all day. Kapacitor’s monitoring and alerts are managed through TICKScripts which can be configured using either the Kapacitor command line client or Chronograf. We have provided sample TICKScripts which cover some common use cases.

Why we moved to the TICK stack

As the Kloudless Platform has grown, the volume and complexity of metrics data has grown with it. Previously, we were using with StatsD as a collector and Graphite to visualize metrics. This simple solution was easy to work with, but our needs have changed. Here are some of the unsupported scenarios we encountered:

  • StatsD doesn’t support tags. Tags are useful for providing context for the measurement, such as status or type of a request. Our workaround was to append tags onto the measurement name, similar to the tags used by DataDog’s DogStatsD. The downside of this approach was messy measurement names that were difficult to query.
  • Each StatsD measurement only has one value. It is sometimes useful to have a tuple of data grouped together in a single metric, or associate data such as application IDs to a metric. This isn’t possible with the StatsD+Graphite solution either.
  • The language used to query Graphite is limited to nesting of functions, and performing aggregations can be slow since the metrics are stored in flat files. This results in slow queries across multiple metrics and prevents easy use of complex operations.

These factors led us to look for a better alternative. InfluxDB seemed promising as its design was tailored for high volume time-series data and metrics collection. InfluxDB is built around measurements, which are in turn made up of many data points. A data point can have one or more values associated with it, and as many tags as needed. This addressed our first two issues right away and allowed for better querying.

For example, let’s try to determine which Kloudless applications were associated with failed API requests in the past day. We can do this with the following query:

select status, path, application_id from request_metrics_api_requests 
where time > now() - 24h and status=~/[^2][0-9]{2}/

In just one query, we’ve selected multiple values (status, path, application_id), filtered by tag values (status is not a 2XX code) and with a time frame limit (last 24 hours). This would have been messier and more tedious with our previous metrics system which did not allow associated metadata to be recorded. Additionally, InfluxDB’s time-series database design allows for tag-based queries to execute more efficiently since all tags are indexed.

Next Steps

We’ve been using the new metrics system at Kloudless to better understand, measure and monitor the performance of our hosted cloud platform. We’ve found that it has saved us time and effort when triaging issues, and we think you’ll find it useful as well. We’re excited to make this metrics system available to our enterprise customers using the Kloudless appliance. Customers using our cloud platform will also see analytics data for their Kloudless applications exposed in the developer portal in the upcoming months. As always, we would love to hear your thoughts and feedback on Twitter, comments below, or at hello@kloudless.com.

Calendar API: Availability Endpoint now… Available!

This post was written by our software engineer, Ryan Connor.

Finding an appropriate meeting time can take a lot of effort, even with just a few participants who have only a single calendar. Finding a meeting time that is sure to work for many participants with multiple calendars managed by several cloud services? Good luck!

Fortunately, Kloudless now helps you do just that. We are proud to announce that the Calendar Availability endpoint is online and ready to help your application find meeting times that work for any combination of user accounts and their calendars.

Using the Calendar Availability Endpoint

The Calendar Availability endpoint returns all available time windows among a set of calendars that match a specific meeting duration and desired time windows for the meeting. The endpoint also supports requests involving multiple calendars for multiple accounts.

To illustrate, imagine your app has 100 users who have two personal (Google) calendars and two work (Outlook) calendars for a total of four hundred calendars. One call to the Calendar Availability endpoint can retrieve available meeting times (if any) for all 100 users subject to all four of their calendars. Alternatively, you can retrieve available meeting times for just a subset of users and/or their calendars.

Kloudless seamlessly integrates with Google and Outlook Calendar behind the scenes—all you have to provide are Account IDs, Calendar IDs, a meeting duration, and time constraints. Let’s take a look at exactly how to do that.

Formatting the Request and Parsing the Response

Here are the details for the request and response format of the Calendar Availability endpoint based on our docs. You can scroll down further to see concrete examples.

Request Format

URL: https://api.kloudless.com/v1/accounts/{account_id,account_id,…}/cal/availability

Method: POST

Headers:

  • Authorization: Bearer [Token]
  • Content-Type: application/json

Body:

  • calendars: List of Calendar IDs. Uses the default calendar if empty. (Optional)
  • meeting_duration: ISO 8601 format for duration. (Required)
  • constraints: A dictionary of constraint keys and values. (Required)
    • time_windows: List of desired time slots with the following values.
      • start: ISO 8601 datetime format
      • end: ISO 8601 datetime format

Response Format

Headers:

  • Content-Type: application/json

Body:

  • time_windows: List of desired time slots with the following values.
    • start: ISO 8601 datetime format
    • end: ISO 8601 datetime format

The start and end times of each time_window in the response bookend (inclusively) the time periods in which all of the accounts are available given the constraints. Note that the times are returned in the GMT time zone, so you may want to convert to the time zone of your choice.

Concretely, if the response includes the “2 – 5 PM” time window and you wanted a 30-minute meeting, you can safely schedule the meeting to start and end at any time within 2 – 5 PM, inclusive (e.g., 2 – 2:30, 2:01-2:31, …, 4:29-4:59, 4:30-5:00).

Example Usage

Single account, two calendars specified

curl -H 'Authorization: Bearer [TOKEN]' \
    -H 'Content-Type: application/json' \
    -XPOST -d '{
        "calendars": ["ra5werWRsZXQzLcRik5BudGltb6RwwUNnbWFpbC5jb21=",
                      "fa2xvdWRsZXNzLnRlc3QudGltb3RoeUBnbWFpbC5jb20=”],
        "meeting_duration": "PT1H",
        "constraints": {
            "time_windows": [{
                "start": "2017-05-20T08:00:00+07:00",
                "end": "2017-05-20T12:00:00+07:00"
            },{
                "start": "2017-05-21T08:00:00+07:00",
                "end": "2017-05-21T12:00:00+07:00"
            }]
        }
    }' \
    'https://api.kloudless.com/v1/accounts/123/cal/availability'

 

{
  "time_windows": [
    {
      "start": "2017-05-20T02:00:00Z",
      "end": "2017-05-20T04:00:00Z"
    },
    {
      "start": "2017-05-21T03:00:00Z",
      "end": "2017-05-21T04:00:00Z"
    }
  ]
}

In this case, the requestor wants to find an appropriate meeting time for a one hour meeting during the 8 AM – 12 PM GMT+7 time window on either May 20, 2017 or May 21, 2017. The requestor is requiring one account as a participant in the meeting and further specifying two calendars from that account.

Note that the primary calendar within an account is automatically considered if no calendars are provided. But, if any calendars are provided, the primary calendar must be included to be considered. For example, if the primary calendar ID in this case is neither ra5werWRsZXQzLcRik5BudGltb6RwwUNnbWFpbC5jb21= nor fa2xvdWRsZXNzLnRlc3QudGltb3RoeUBnbWFpbC5jb20=, the primary calendar would be ignored when retrieving availability. To also consider the primary calendar, the requestor should provide it as a third calendar in the given calendar list.

The response indicates that any one-hour slot between either 9 AM – 11 AM GMT+7 on May 20, 2017 or 10 AM – 11 AM on May 21, 2017 works for the meeting. Of course, the desired meeting length is exactly as long as the boundaries of the second available time window, so there is only one slot that works on that date.

Multiple accounts, no calendars specified

curl -H 'Authorization: Bearer [TOKEN]' \
    -H 'Content-Type: application/json' \
    -XPOST -d '{
        "meeting_duration": "PT45M",
        "constraints": {
            "time_windows": [{
                "start": "2017-06-15T08:00:00-03:00",
                "end": "2017-06-15T17:00:00-03:00"
            }]
        }
    }' \
    'https://api.kloudless.com/v1/accounts/123,456,789/cal/availability'
{
  "time_windows": [
    {
      "start": "2017-06-15T05:00:00Z",
      "end": "2017-06-15T06:30:00Z"
    },
    {
      "start": "2017-06-15T07:45:00Z",
      "end": "2017-06-15T10:00:00Z"
    },
    {
      "start": "2017-06-15T12:30:00Z", 
      "end": "2017-06-15T14:00:00Z"
    }
  ]
}

Here, the requestor wants to find a time for a 45-minute meeting during the 8 AM – 5 PM GMT-3 time window on June 15, 2017. The requestor is requiring three accounts as participants in the meeting, and since no calendars are specified, is searching for availability based solely on each account’s primary calendar.

Note that you can pass time window start and end times in any time zone (and, in fact, differing time zones within the same request). Furthermore, the calendars to be searched on need not be in the same time zone as the constraints. The important point is that the times are passed as ISO 8601-formatted strings—Kloudless will handle the rest.

The response indicates that any 45-minute slot between 9 AM – 10:30 AM GMT-3, 0:45 AM – 1 PM GMT-3, or 3:30 – 5 PM GMT-3 on June 15, 2017 works for the meeting.

Looking to the Future

The Calendar Availability endpoint provides a simple way to coordinate meetings among multiple calendars and calendar accounts. Going forward, we will monitor usage of the endpoint and listen to your feedback to determine the best way to expand the endpoint’s functionality and flexibility. We already have some great suggestions for making the endpoint more customizable, including:

  • The start_times constraint. If the requestor provides start_times in addition to (or in lieu of) time_windows, the endpoint would return the subset of start_times that work as the start time for the meeting. Essentially, this lets the requestor ask “Which of these exact start times are OK for the meeting?”
  • The ​min_attend_percent constraint. Right now, a time window will only be included in the response if every given account is available in that window—that is, if one relevant calendar in one account is not available during a window, that account is not available and so that window is not returned. With the min_attend_percent constraint, a time window would be included if at least the specified percentage of accounts are available during that window.e
  • The work_or_personal constraint. The requestor could provide a work_or_personal argument to limit the times considered for work or personal hours. This could be especially useful when trying to coordinate across widely varying time zones. For example, a colleague across the world may technically be “available” for a meeting at the time the requestor desires, but that’s of little use if it’s 2 AM for that colleague!

Wrapping Up

We’re excited to rollout the Calendar Availability endpoint to the developers on our platform. What do you like about the endpoint right now, what questions do you have about it, and what do you wish it would do in the future? We would love to hear your thoughts and feedback on Twitter, our developer forums, comments below, or at hello@kloudless.com.

 

Calendar API: Activity Stream

This post is by our engineering intern, Ellen Luo.

Receive real-time updates for Google and Outlook Calendar with the new Events endpoint for the Calendar API!

The Events endpoint allows your application to easily keep track of calendar items that have been added, modified or deleted in your primary calendar. This Events endpoint is part of our broader Events API to monitor activity in connected accounts, and should not be confused with the Calendar Events API that enables access to calendar appointments.

Retrieving activity data with the Events API

To start using the Events API, check the box beside “Collect Events” in the App Details page of the Developer Portal. Kloudless will automatically begin to collect activity data for any newly connected Google or Outlook Calendar accounts. Google Calendar allows real-time updates of changes to users’ accounts so event data is updated in at most 1.5 minutes. For Outlook Calendar, Kloudless queries the service repeatedly to find changes. This means that event data will be updated on average every 15 minutes in our cloud and 1-5 minutes in private installs (configurable).

Requesting activity data

To return a list of all recent events collected, make a request to https://api.kloudless.com/v1/accounts/{account_id}/events/ with your connected account’s ID and Bearer token (or application API Key). A sample request is shown below:

curl -X GET -H 'Accept: application/json' \
    -H 'Authorization: Bearer {token}' \
    'http://api.kloudless.com/v1/accounts/{account_id}/events/'

Parsing the response

The response will contain a list objects of all the events. Each object has a type, which indicates the type of change. Right now, the only events that are detected are “add” (object created), “update” (existing object modified), and “delete” (object deleted). Each event object will have the updated metadata of the object modified. More information about the Events API and sample responses can be found in the Kloudless docs here.

Roadmap

In the future, we will add support for retrieving activity data for calendars other than the account’s primary calendar. We will also be working towards including more specific event types and expanding support to more calendaring APIs. We’re excited to make the Calendar API Activity Stream endpoint available to all developers on our platform and would love to hear any feedback or suggestions on Twitter, the comments below, or at hello@kloudless.com.