How to connect with 1.2 billion Microsoft users

Last week at the Connect(); conference in New York, Microsoft publicly unveiled the Microsoft Graph, a unified API that enables developers to tap into Microsoft’s 1.2 billion users around the world.

There is a lot going on in the Microsoft Graph, but we will look closely at the file storage services: OneDrive (OD), OneDrive for Business (ODB), and SharePoint Online (SP).

What is the Microsoft Graph

Microsoft Graph is a unified API for connecting to Office 365.

Previously, each Office 365 service was silo’d and had its own unique API. This required developers to obtain separate access tokens and call different endpoints for each service they wanted to integrate with.

With the new unified API, features, data, and insights across Office 365 services can all be accessed from a single API.

That’s nice but..

Historically, Microsoft has not been great with APIs. Services often have multiple versions, each with multiple APIs spanning hundreds of pages of documentation.

Our implementations of OD, ODB, and SP pull from several different APIs in order to get comprehensive feature and data coverage. Of course, developers that use Kloudless are shielded from this complexity, since we offer a universal interface for integrating file storage services.

Getting started

If you’re not a Kloudless developer yet, this means that you’ll have to go through another proprietary API, integrate it into your application, and maintain it in the long run. You can read through the documentation and try out the Microsoft Graph at

If you have an existing integration with Office 365 service, make sure to check for feature parity between the old APIs and the new unified API before overhauling you current implementation.

We noticed several differences. Some features like adding an admin user as a site collection administrator SP are only found in the old SP APIs. Other features like listing all site collections (SP) and personal sites (ODB) are only available in the new Microsoft Graph API.

For use cases that only require basic operations, you’ll be fine switching over to the new API. Otherwise, like us, there’s a good chance you’ll find yourself mixing and matching APIs to get everything you need.

If you are using Kloudless, you’re already integrated with the Microsoft Graph! We’re dedicated to bringing you the latest and greatest, so we worked with Microsoft earlier this year to get our hands on a preview of a couple unreleased APIs, including the Office 365 Unified API.

As a result, no extra work is required to transition to the Microsoft Graph, because we’ve taken care of everything under the hood. You will automatically receive all the improvements and new features.

New features supported in Kloudless

You can access new Microsoft features via the respective Kloudless endpoints.

Events and Notifications (ODB, SP) – Real-time notifications for what’s changed in a folder.

Permissions (OD, ODB, SP) – Adding, modifying, and removing permissions on files and folders. (Coming soon)

What we think about the new Graph

We like all the talk coming out of Redmond. Unifying APIs is a mission that we can get behind.

However, a few challenges remain if you’re looking to integrate file storage APIs into your application:

  1. Uniform APIs are a great promise, but execution and timeliness matters. A few years in, OneDrive and OneDrive for Business still have many API differences.
  2. It sounds like a lot of updates are planned. If you plan on integrating directly with the new API, be prepared for several rounds of maintenance work in the near future.
  3. The Microsoft Graph will not unify the other 25+ file storage APIs like Box, Dropbox, Google Drive, Egnyte, etc.

This is definitely a step in the right direction for Microsoft to endear themselves with developers. It certainly makes our job a bit easier.

Kloudless unifies 25+ file storage services under a universal API. If you’d like to easily integrate the Microsoft services and many others, sign up for a Kloudless account and start developing for free!

Level Up: A Guide to Kloudless Enterprise Clustering (III of III)

This blog post was authored by David Thorman, who leads Ops at Kloudless.

In our last post we covered the different options for deploying Kloudless Enterprise. When deploying Kloudless Enterprise, you would start off with a single server. However, when running any application, there is a limit to how much work a single server can handle. A server can be scaled vertically by allocating more CPU cores and memory, but eventually those limits will be reached and another mechanism to increase capacity must be found. This is where clustering is useful.

Clustering allows your application to scale out horizontally rather than up which leads to two benefits: higher throughput capacity and high availability. Clustering is made possible by running more than one Kloudless Enterprise instance behind a load balancer. A load balancer is a server that accepts requests from clients and then forwards them to the backend Kloudless Enterprise servers. This allows the client to take advantage of multiple backend servers without having to manually keep track of their hostnames. The distribution of work allows the cluster to handle a higher number of requests than a single server would, resulting in greater throughput.

Screen Shot 2015-09-28 at 8.55.00 PM

The cluster can continue to serve requests in the event of a failed secondary instance, ensuring high availability. If the primary instance fails, a secondary instance will be promoted to primary, ensuring smooth disaster recovery. Each node is capable of handling similar work, and is configured to be the same size. This means the failure of a single node will not result in the cluster not being able to serve requests.

Service interruption can be minimized by ensuring that the load balancer only sends requests to nodes that it realizes are healthy. This is typically achieved via health checks that the load balancer performs. For example, in AWS, the Elastic Load Balancer’s health checks take the form of an HTTP request to each individual node in the cluster. Nodes that either don’t return or return a non-200 status code are marked as unhealthy. This allows the cluster to handle requests successfully even though there are failures.

Thanks to the HA/DR features above, the cluster can be dynamically scaled up or down without interrupting service to clients. Adding new nodes increases capacity when your service experiences periods of higher load. Removing nodes allows costs to be reduced during periods of lower load. On certain platforms ,these changes in capacity can be handled automatically either on a timed schedule or based on metrics gathered from the cluster itself. This allows you to take full advantage of the elasticity of modern IaaS platforms such as AWS without disrupting service to your customers.

This post has covered the high-level benefits of clustering and the flexibility of a Kloudless Enterprise deployment. Our configuration guide covers the technical details of deploying Kloudless Enterprise as well as a walkthrough detailing how to deploy Kloudless Enterprise in an auto-scaling cluster. The guide can be accessed by emailing us at Feel free to reach out to us or comment below with any questions.

Thanks for following our Kloudless Enterprise Series!

Level Up: You Call the Shots (Part II of III)

In our last post, we introduced Kloudless Enterprise, a version of our software that you can host in your own private infrastructure.

There are actually several options for deploying Kloudless: Cloud, Cloud Private Install, On-premises, and a couple more that we can’t talk about yet. But, we get it. Too many choices, too little time.

Here’s a side-by-side comparison of our current deployment options to help you quickly pick the options that’s right for you:


Private Install


Manager Kloudless Kloudless You
Hosted in AWS AWS
Your or your customer’s private infrastructure
Connectors Cloud services only Cloud services only Cloud services + On-premises proxy connector
Scaling Auto-scaled based on total usage across all tenants Auto-scaled based on your usage Scaling managed by you
Availability Kloudless manages high availability Kloudless manages high availability via Enterprise Clustering. Clustering support to enable high availability
Security Data encrypted in transit and at rest. 24/7 Ops team on call Same as “Cloud” except your data is isolated. Access is restricted to your IPs Inherits your security and regulatory compliance promises. No data passes through Kloudless infrastructure

The Level Up mini series is wrapping up soon –don’t miss it! If you can’t wait until our next blog post and want to start using Kloudless Enterprise today, drop us a line at

Level Up: A Developer’s Intro to Kloudless Enterprise (Part I of III)

Get all the benefits of Kloudless without using Kloudless servers.

That’s right. Now, you choose where your data goes. Introducing, the latest version of Kloudless: Kloudless Enterprise.

Kloudless Enterprise is a virtual appliance that is installed to private infrastructure. This means that you can host Kloudless in your own private cloud or on-premises data center.

Unlike the Cloud or Cloud Private Install versions of our service, the data exchanged between your application and storage services never passes through Kloudless’s infrastructure. Your servers communicate directly with the file storage services.

In the next two blog posts, we’ll cover how Kloudless Enterprise differs from our Cloud Private Install and the security/performance benefits of going on-premises.

On Premises Illustration
Stay tuned for Part II! Interested in hosting Kloudless Enterprise on your own servers? Shoot us an email at

VIP Kloud Treatment

If you built and deployed an application without contacting us, you are using our Cloud version, which is hosted on Amazon Web Services, multi-tenant, and fully managed by us. However, as the saying goes, “too many chefs in the kitchen can ruin the broth.” (This isn’t the best analogy, plus only we can ruin our broth, but you get the point.).

For a number of reasons (data security, special support requirements, etc.), not all developers want to be part of a multi-tenant facility. That’s why we’re introducing our new VIP service: the Cloud Private Install.

Cloud Private Installs are hosted on an instance in a service provider of your choice (AWS, Azure, Rackspace, etc.), with all features from the Cloud version supported. The installation is fully managed by Kloudless, including all infrastructure requirements, security updates and application upgrades. Based on the performance and availability you need, we can come up with the right instance category and bring up the virtual appliance in the appropriate regions of the cloud provider of your choosing.

Benefits of the Cloud Private Install include:

  • Superior networking performance and data throughput (e.g. Up to 10 Gbps on AWS)
  • Multi-tenant capable: CPU affinity can be optimized for either single or multiple tenants
  • Complementary proxy tool, enabling you to connect to on-premises storage repositories in your customer’s infrastructure
  • Custom SLAs and higher tiers of support

Our Cloud version and Cloud Private Install come with 99.9% uptime and a dedicated Ops team available 24/7, so regardless of which edition you’re using, you’re in good hands.

Interested in becoming a VIP? Get started with a Kloudless developer account and drop us a line at

Kick Size Restrictions to the Curb

With improved Multipart and URL uploads, we can kiss goodbye to file size restrictions and say hello to efficiency.

URL Uploads

URL uploads enable files to be obtained from a URL and uploaded to a location in cloud storage. Previously, file sizes were limited to only 100 MB. Now, URL uploads can support file sizes greater than 100 MB via an automatic multipart upload.

Multipart Uploads

Multipart Uploads can directly upload large files, and now support seven additional cloud services: Azure, Amazon S3, Dropbox, Egnyte, Google Drive, OneDrive, ShareFile, SharePoint and OneDrive for Business.

Uploading in parts minimizes the consequences of failed uploads due to connection issues. Several cloud services require that files over a certain size be uploaded in parts. Developers can also parallelize uploads with Azure and Amazon S3, with support for parallel uploads to other services coming soon.

This update isn’t the last of it –follow us on Twitter and Facebook for more!

Upgrade your Toolbox: SDKs and File Explorer

Kloudless developers have a new Ruby SDK, updated Python and Java SDKs, as well as an improved File Explorer at their fingertips.


A Ruby SDK is now available! Try it out here.

The Python and Java SDKs have been upgraded to include the Team and Permissions endpoints, as well as the Management API.

  • The Team endpoint allows developers to easily request and retrieve information on their Users and Groups.
  • The Permissions endpoint grants seamless management to the developer for user access and rights to their data.
  • The  Management API enables developers to conveniently control their Applications and API Keys.

You can access the Python SDK and Java SDK on Github, along with the rest of the Kloudless UI toolkit.

File Explorer

If you can recall, we open-sourced our File Explorer last October, but we’re not stopping there.

We’ve expanded the File Explorer’s events and methods. Check out the full list here.

The File Explorer now supports a search bar, allowing for users to easily search for and find files in an account. Improved sorting capabilities also aid in easily locating a file.

Feel free to leave us any questions, comments, or feedback regarding the Kloudless UI Toolkit by sending us an email. Not a part of the Kloudless family yet? Sign up for your free developer account here.

Getting Started with Kloudless Webhooks


There are times when your application needs to keep up with changes that take place in your users’ cloud storage. Perhaps your application needs to perform an action when new files appear, or needs to update its state when files are modified/deleted. Kloudless solves this problem by making an HTTP request (a “webhook“) to a URL you define, with information on which accounts have changed. You can then query the events endpoint for more detailed information on modified files in the account.

The first release of Kloudless’ events endpoint supports Dropbox, Box, OneDrive, Evernote, Sharepoint, and OneDrive for Business. While some services like Box expose an endpoint for listing events, others such as SharePoint and OneDrive do not. Kloudless supports these services by periodically polling them for changes, allowing you to have a consistent interface to query all the services for changes.

This short walkthrough gets you up and running with Kloudless’s Webhooks. It assumes that:

  • You already have a Kloudless account and app set up (if you don’t, you can get one here)
  • You know some Python (we will be using Flask to create a basic webserver)
  • You are familiar enough with HTTP to make/understand requests
  • You can run code somewhere accessible by the Kloudless server requests

Configuring Webhooks

The webhooks for your application can be configured on the Application Details page of the developer dashboard. All you have to do is add the URL that you would like Kloudless to make requests to. Note: before the webhook is saved, Kloudless makes a test request to it to ensure that it works, so if your application isn’t up and running yet, you won’t be able to create the webhooks.


The Structure of the Webhooks

The requests that we make for our webhooks are extremely simple. Here is the raw HTTP of an example request Kloudless might make to the above endpoint:

POST /kloudless-webhook HTTP/1.1
Content-Length: 14
Accept-Encoding: gzip, deflate, compress
Accept: */*
User-Agent: kloudless-webhook/1.0
X-Kloudless-Signature: VPl14EUveHQ7HcAvigiRZ7dmX7QLM5iD+pnXJv2XYok=
Content-Type: application/x-www-form-urlencoded


As you can see it is a standard http form. We provide the id of the account that we have new events for. There will only be one account per notification. The primary purpose is to inform you that new events are available from the events endpoint. We also include a signature so that you can verify that the event actually came from our service.

Our service expects a 200 OK, with the response body consisting of only your Kloudless Application ID (available on the Developer Portal’s App Details page).

A Simple WebHooks Receiver

Using Flask we can quickly build a simple web server that can receive these notifications, retrieve the corresponding events, and do something interesting. For an extremely simple server you can use for testing purposes, check out this Gist.

Here, we will describe a more useful receiver that verifies the webhook sender and launches a task. We start with the event receiver:

#!/usr/bin/env python
from flask import Flask, request, abort

import hmac
import base64
import hashlib
import threading

app = Flask('demo_app')
APP_ID = ''
API_KEY = ''
@app_route('/webook', methods=['POST'])
def receiver():
# Verify the Request Signature
digest =,, digestmod=hashlib.sha256).digest()
sig = base64.b64encode(digest)
if not sig == request.headers.get('X-Kloudless-Signature'):

# Since the request is verified we can actually do something
account_id = request.form.get('account')
if account_id:
threading.Thread(target=process_account, args=(account_id,)).start()
# In order to verify that the endpoint is alive you should always return your application id with a 200 OK response
return APP_ID

if __name__ == 'main':

This basic structure allows you to insert just about any functionality into the process_account function. So that you don’t fetch the same events over and over again, you will have to keep track of where you are in the event stream for each account. Each time you make a request to the events endpoint you get back a cursor value that marks your current place. To track our current progress through the event stream, we are going to store the cursors for each account id in [redis])( You could use your preferred data store, but redis is simple and quick to get started with. If you want to fetch new events and then process added files, write a function similar to the following (this uses python requests so that it is easier to translate to other languages):

import requests
import redis

redis_client = redis.StrictRedis(host='localhost', port=6379, db=0)

def process_account(account_id):
# if there is an existing cursor we want to make sure we use it
cursor = redis_client.get(account_id)
params = {"cursor": cursor}
headers = {"Authorization": "ApiKey %s" % API_KEY}
url = "" % account_id
response = requests.get(url, headers=headers, params=params)

if response.ok:
data = response.json()
events = data['objects']
new_cursor = data['cursor']
for event in events:
if event['type'] == 'add': # Only care about new/modified files
# updating cursor in datastore
redis_client.set(account_id, new_cursor)

def something_cool(file_id):
print "There is a new file: %s" % file_id

In this case, you would put your interesting functionality in the do_something_cool function. For example, you could do some sort of document conversion or content auditing with the file id that you get. The event contains the full file metadata so you can make other decisions about what to do based on extension, parent folder, or mime-type.

Best Practices

There are a few things that should be done to ensure that your Kloudless Webhooks integration works as well as possible.

Handle WebHooks Quickly

The initial handling of the webhooks requests should be done as quickly as possible. Otherwise, a high volume of requests could easily overwhelm a naive implementation that does a large amount of work prior to sending a response to the webhook requests.

Manage Concurrency

When there are lots of changes occurring for a particular account, a large number of notifications could be sent to your server. Your application should make sure that this doesn’t cause problems. For certain types of applications, the duplicate actions would not cause any issues. However, if you have non-idempotent actions or actions that are particularly expensive in terms of computation time, then make sure there is locking, leasing, or mutual exclusion for that user for the duration of the action. This ensures that duplicate work isn’t being done while the action is running.

There are other design practices for event-driven applications, but they are beyond the scope of this post.


Hopefully this will have provided you with a good introduction to how to take advantage of Kloudless Webhooks so that you can build more effective event-driven applications around your user’s cloud storage. Please email or chat with us on Stack Overflow if you have any feedback, questions, or insights about building your application with Kloudless Webhooks.