Office 365 PowerShell queries via REST: Maximizing the Kloudless Pass-through API

In our previous post, we announced the availability of the Kloudless Pass-through API. The Pass-through API enables your application to make API requests directly to third-party services, while still using Kloudless’s unified APIs. In this blog post, we’ll discuss how to access the Office 365 PowerShell via the Kloudless REST API to perform administrative tasks in Office 365.

Building with our easy-to-use REST API offers many benefits. We handle the complexities of integrating with each service behind the scenes so you  don’t have to. This speeds up your integration time and decreases future maintenance. We’ve extended the same principle to our Pass-through API by introducing special capabilities such as enabling PowerShell queries for Office 365 admin accounts.

Office 365 PowerShell provides several remote management commands that can be used to administer your Office 365 tenant, similar to how you would via the Office 365 admin center web application. For example, a broad set of security and compliance features can be accessed via theSecurity & Compliance Center cmdlets by connecting to Office 365 using remote PowerShell. Normally, you would need access to a PowerShell prompt in order to access this functionality. The Kloudless REST API handles the heavy-lifting and enables you to access this functionality via our REST API.

Invoking Office 365 Security Center cmdlets via the Kloudless Pass-through API

To begin, connect a SharePoint Online admin account to your Kloudless application. The easiest way to do this is by logging into your Kloudless account and then navigating to the Interactive Docs. Click the “Add Account” button and then click on SharePoint Online under the “Admin accounts” section towards the bottom of the pop-up that opens.

idjg03n

Once you’ve connected your account, you will receive a Kloudless Account ID that can be used for API requests to the Kloudless API. You are now ready to make Pass-through API requests to this admin Office 365 account!

While SharePoint Online REST API requests can be performed without any additional configuration, the PowerShell queries described in this blog post require special permission to access. Please contact us at support@kloudless.com if would like this capability enabled for your developer account.

Request Format

The format of PowerShell pass-through API requests is as follows:

URL: https://api.kloudless.com/v1/accounts/{account_id}/raw
  • {account_id} is the Kloudless account ID of the SharePoint Online admin account connected.
Headers (described in the Pass-through API docs):
  • X-Kloudless-Raw-URI: http://powershell/ This special value indicates the request should be translated to a PowerShell query.
  • X-Kloudless-Raw-Method: POST
  • Authorization: Bearer {account_bearer_token} OR Authorization: APIKey {application_api_key} See our Authentication Docs for more information on authorizing API requests.
Body
JSON data in the format below:

    {
      "category": "o365-security",
      "command": {cmdlet_name},
      "options": {
        ... option name: value mappings if required ...
      }
    }
At the current time, only Office 365 Security and Compliance Center cmdlets ("category": "o365-security") and Exchange Online cmdlets ("category": "exchange") are available via the Kloudless API. If you would like access to other remote PowerShell cmdlets, please contact us at support@kloudless.com.

Examples of Requests

An example of a curl request with the format described above would be:

curl -H "Authorization: APIKey {api_key}" \
     -H "X-Kloudless-Raw-URI: http://powershell/ \
    https://api.kloudless.com/v1/accounts/{account_id}/raw \
    --data '{body}'

Please replace the {api_key}, {account_id} and {body} values with your API Key, connected account’s ID and JSON data for PowerShell respectively.

Here are some examples of Body data to use in {body} for specific cmdlets:

Get-ComplianceCase
Obtaining a list of compliance cases.

{
  "category": "o365-security",
  "command": "Get-ComplianceCase"
}
An example of a curl request for this would be:

curl -H "Authorization: APIKey 123ABC" \
     -H "X-Kloudless-Raw-URI: http://powershell/" \
     https://api.kloudless.com/v1/accounts/123/raw \
     --data '{"category": "o365-security", "command": "Get-ComplianceCase"}'
New-ComplianceCase
Create a new compliance case.

{
  "category": "o365-security",
  "command": "New-ComplianceCase",
  "options": {
    "Name": "test new case 2",
    "Description": "This case is created via curl"
  }
}
An example curl request would be identical to the one used for Get-ComplianceCase but with the new value above for --data instead.
New-CaseHoldPolicy
Create a new hold policy for a case.

{
  "category": "o365-security",
  "command": "New-CaseHoldPolicy",
  "options": {
    "Case": "3b4de8d5-13cb-4291-bdd0-b6e2bb82a08e",
    "Name": "New Hold",
    "SharePointLocation": "https://kloudless.sharepoint.com/test subsite/"
  }
}
where "3b4de8d5-13cb-4291-bdd0-b6e2bb82a08e" is the Identity GUID of the Case to add the legal hold policy to. This corresponds to the Locations section of a Hold when editing a Case’s Holds at https://protection.office.com/#/ediscovery.
New-CaseHoldRule
Create a rule to add to a hold policy.

{
  "category": "o365-security",
  "command": "New-CaseHoldRule",
  "options": {
    "Policy": "New Hold",
    "Name": "New Rule",
    "ContentMatchQuery": "SSN"
  }
}
where "New Hold" is the name of the hold I created previously. This corresponds to the Conditions section of a Hold.

Similarly, other policies such as retention policies can also be created, and existing objects can be deleted:

New-RetentionCompliancePolicy
Create a new preservation policy.

{
  "category": "o365-security",
  "command": "New-RetentionCompliancePolicy",
  "options": {
    "Name": "Test new policy",
    "SharePointLocation": "https://kloudless.sharepoint.com/"
  }
}
This creates a policy but has not yet created a rule to add to it, which can be done with .
Remove-ComplianceCase
Deleting a compliance case.

{
  "category": "o365-security",
  "command": "Remove-ComplianceCase",
  "options": {
    "Identity": "94b99324-5574-4220-b081-1b689cb386af",
    "Confirm:$false": null
  }
}
where "94b99324-5574-4220-b081-1b689cb386af" is the Identity GUID of the Case to remove.

Future capabilities of the Pass-through API

The Pass-through API provides provides a powerful new way to access third-party features via Kloudless as shown above. We’re excited to make this available to all developers on our platform and would love to hear any feedback or suggestions in our developer forum.

Announcing the Kloudless Pass-through API

We are excited to announce the general availability of the new Kloudless Pass-through API endpoint. The Pass-through API enables your application to access the full functionality of the third-party software service by performing API requests directly to the service. Kloudless continues to ensure any relevant authentication information like refreshed access tokens are included in the API request.

While Kloudless provides unified APIs, there may be times when your application needs to access unique functionality in a third-party service that is not available in Kloudless’s unified APIs. The Pass-through API solves this problem by enabling your application to send raw data to a third-party API’s endpoint and receive the raw response streamed directly from the service through Kloudless.

Check out our API docs for more information on how to use the Pass-through API.

Kloudless API v1 Launches Today

Last week, we launched our Universal CRM API. You can read more about the announcement here. The new API kicks off our broader product strategy to provide Universal APIs for many important areas of business.

As part of this product launch, we are also launching v1 of our API. Our goal with v1 is to establish a strong foundation for building future products and functionalities.

 

What’s new in v1

The migration to v1 introduces new capabilities and improvements to existing functionality.

Universal CRM API
Previously, the Universal CRM API was only available in Kloudless Enterprise. With v1, the CRM API is available to any developer that prefers to use our cloud version at kloudless.com. You can start integrating the CRM API for free here.

OAuth 2.0
We are introducing a standards-compliant OAuth 2.0 authentication mechanism to connect user accounts, which enables easy integration with other tools that support OAuth 2.0. Other benefits include:

  • Documented support for out-of-band, authorization code grant, and implicit grant OAuth 2.0 flows to support authentication in any environment.
  • More granular scopes to better control the services, APIs and types of accounts users can connect to your app.
  • Improved security with white-listed Redirect URIs, validated Client IDs/Secrets, and verified Bearer tokens.

Read more about Kloudless authentication here.

Other enhancements
As you read through our documentation, you will find small improvements to many endpoints. For example, we’ve updated our old permissions format to support groups and our old pagination format to be more flexible.

 

What this means for you

First, any new applications you develop should use API v1.

Second, there are backwards incompatible changes in the migration from v0 to v1. API v1 is built thoughtfully, and we try to limit the changes you need to your implementation to only the things we consider most important moving forward. You should update your app’s integration with Kloudless as soon as possible.

Here is a summary of the backwards-incompatible changes.

API-namespacing for endpoints
All endpoints specific to the Storage API have been namespaced under /storage/. Learn more

OAuth 2.0 Authentication
OAuth 2.0 authentication is now required. The previous authentication format is no longer supported. OAuth 2.0 Bearer tokens have replaced Account Keys. Learn more

Events
Deprecated event object attributes have been removed. Learn more

Pagination
next_page must be used to identify the next page value for pagination. Learn more

File Uploads
Files are uploaded as binary content rather than via multipart form POST requests. Learn more

Permissions
File/Folder permission updates take in a list of permission objects rather than a mapping of user emails to roles. Learn more

Users of the UI Tools do not need to make changes to use v1, although they would have to make changes to switch to OAuth if using the Authenticator JS library or if using Account Keys from the File Explorer.

None of your users need to re-authenticate when you switch your app to either OAuth 2.0 or the v1 API.

 

Deprecating v0 and the old auth mechanism

As of today, v0 of the API is deprecated. In order to provide you with the most up-to-date features and support a single, consistent API platform, we’ll be turning off API v0 on February  28, 2018.

Our previous authentication mechanism will be turned off in one year on August 31, 2017.

In our next update, we will provide a more detailed deprecation timeline.

We love building for the thousands of developers using our platform. If you have any questions or feedback, we’d love to hear from you in our developer forum.

An Eventful Update

Kloudless developers can now manage their events even more efficiently using the new Events Endpoint updates. Check out what our engineers have been tinkering with below!

Kloudless Enterprise Events

Connect your Admin account and get access to organization-wide events. Enterprise Events can obtained through the normal events endpoint. The user responsible for the event is specified where applicable. 

Events Endpoint Pagination

The Events endpoint now supports requests of a specific page size and also returns the number of remaining events. It also supports only the retrieval events created after the cloud account has been connected to the Kloudless application. Additionally, a more granular list of event types is also now available, instead of + and -.

S3 Event Notifications

Event data and webhook notifications are now available for changes to data in S3 accounts. Any S3 accounts requiring this feature must be reconnected.

Whether you’re using the cloud, private installs, or Enterprise version of Kloudless, this new update enables your application to respond to activity in cloud storage more effectively.

Not a Kloudless developer yet? Click here to get started. Questions or feedback? Feel free to drop a line at hello@kloudless.com

How to connect with 1.2 billion Microsoft users

Last week at the Connect(); conference in New York, Microsoft publicly unveiled the Microsoft Graph, a unified API that enables developers to tap into Microsoft’s 1.2 billion users around the world.

There is a lot going on in the Microsoft Graph, but we will look closely at the file storage services: OneDrive (OD), OneDrive for Business (ODB), and SharePoint Online (SP).

What is the Microsoft Graph

Microsoft Graph is a unified API for connecting to Office 365.

Previously, each Office 365 service was silo’d and had its own unique API. This required developers to obtain separate access tokens and call different endpoints for each service they wanted to integrate with.

With the new unified API, features, data, and insights across Office 365 services can all be accessed from a single API.

That’s nice but..

Historically, Microsoft has not been great with APIs. Services often have multiple versions, each with multiple APIs spanning hundreds of pages of documentation.

Our implementations of OD, ODB, and SP pull from several different APIs in order to get comprehensive feature and data coverage. Of course, developers that use Kloudless are shielded from this complexity, since we offer a universal interface for integrating file storage services.

Getting started

If you’re not a Kloudless developer yet, this means that you’ll have to go through another proprietary API, integrate it into your application, and maintain it in the long run. You can read through the documentation and try out the Microsoft Graph at graph.microsoft.com.

If you have an existing integration with Office 365 service, make sure to check for feature parity between the old APIs and the new unified API before overhauling you current implementation.

We noticed several differences. Some features like adding an admin user as a site collection administrator SP are only found in the old SP APIs. Other features like listing all site collections (SP) and personal sites (ODB) are only available in the new Microsoft Graph API.

For use cases that only require basic operations, you’ll be fine switching over to the new API. Otherwise, like us, there’s a good chance you’ll find yourself mixing and matching APIs to get everything you need.

If you are using Kloudless, you’re already integrated with the Microsoft Graph! We’re dedicated to bringing you the latest and greatest, so we worked with Microsoft earlier this year to get our hands on a preview of a couple unreleased APIs, including the Office 365 Unified API.

As a result, no extra work is required to transition to the Microsoft Graph, because we’ve taken care of everything under the hood. You will automatically receive all the improvements and new features.

New features supported in Kloudless

You can access new Microsoft features via the respective Kloudless endpoints.

Events and Notifications (ODB, SP) – Real-time notifications for what’s changed in a folder.

Permissions (OD, ODB, SP) – Adding, modifying, and removing permissions on files and folders. (Coming soon)

What we think about the new Graph

We like all the talk coming out of Redmond. Unifying APIs is a mission that we can get behind.

However, a few challenges remain if you’re looking to integrate file storage APIs into your application:

  1. Uniform APIs are a great promise, but execution and timeliness matters. A few years in, OneDrive and OneDrive for Business still have many API differences.
  2. It sounds like a lot of updates are planned. If you plan on integrating directly with the new API, be prepared for several rounds of maintenance work in the near future.
  3. The Microsoft Graph will not unify the other 25+ file storage APIs like Box, Dropbox, Google Drive, Egnyte, etc.

This is definitely a step in the right direction for Microsoft to endear themselves with developers. It certainly makes our job a bit easier.

Kloudless unifies 25+ file storage services under a universal API. If you’d like to easily integrate the Microsoft services and many others, sign up for a Kloudless account and start developing for free!

Level Up: A Guide to Kloudless Enterprise Clustering (III of III)

This blog post was authored by David Thorman, who leads Ops at Kloudless.

In our last post we covered the different options for deploying Kloudless Enterprise. When deploying Kloudless Enterprise, you would start off with a single server. However, when running any application, there is a limit to how much work a single server can handle. A server can be scaled vertically by allocating more CPU cores and memory, but eventually those limits will be reached and another mechanism to increase capacity must be found. This is where clustering is useful.

Clustering allows your application to scale out horizontally rather than up which leads to two benefits: higher throughput capacity and high availability. Clustering is made possible by running more than one Kloudless Enterprise instance behind a load balancer. A load balancer is a server that accepts requests from clients and then forwards them to the backend Kloudless Enterprise servers. This allows the client to take advantage of multiple backend servers without having to manually keep track of their hostnames. The distribution of work allows the cluster to handle a higher number of requests than a single server would, resulting in greater throughput.

Screen Shot 2015-09-28 at 8.55.00 PM

The cluster can continue to serve requests in the event of a failed secondary instance, ensuring high availability. If the primary instance fails, a secondary instance will be promoted to primary, ensuring smooth disaster recovery. Each node is capable of handling similar work, and is configured to be the same size. This means the failure of a single node will not result in the cluster not being able to serve requests.

Service interruption can be minimized by ensuring that the load balancer only sends requests to nodes that it realizes are healthy. This is typically achieved via health checks that the load balancer performs. For example, in AWS, the Elastic Load Balancer’s health checks take the form of an HTTP request to each individual node in the cluster. Nodes that either don’t return or return a non-200 status code are marked as unhealthy. This allows the cluster to handle requests successfully even though there are failures.

Thanks to the HA/DR features above, the cluster can be dynamically scaled up or down without interrupting service to clients. Adding new nodes increases capacity when your service experiences periods of higher load. Removing nodes allows costs to be reduced during periods of lower load. On certain platforms ,these changes in capacity can be handled automatically either on a timed schedule or based on metrics gathered from the cluster itself. This allows you to take full advantage of the elasticity of modern IaaS platforms such as AWS without disrupting service to your customers.

This post has covered the high-level benefits of clustering and the flexibility of a Kloudless Enterprise deployment. Our configuration guide covers the technical details of deploying Kloudless Enterprise as well as a walkthrough detailing how to deploy Kloudless Enterprise in an auto-scaling cluster. The guide can be accessed by emailing us at hello@kloudless.com. Feel free to reach out to us or comment below with any questions.

Thanks for following our Kloudless Enterprise Series!

Level Up: You Call the Shots (Part II of III)

In our last post, we introduced Kloudless Enterprise, a version of our software that you can host in your own private infrastructure.

There are actually several options for deploying Kloudless: Cloud, Cloud Private Install, On-premises, and a couple more that we can’t talk about yet. But, we get it. Too many choices, too little time.

Here’s a side-by-side comparison of our current deployment options to help you quickly pick the options that’s right for you:

Cloud

Private Install

Enterprise

Manager Kloudless Kloudless You
Hosted in AWS AWS
Your or your customer’s private infrastructure
Connectors Cloud services only Cloud services only Cloud services + On-premises proxy connector
Scaling Auto-scaled based on total usage across all tenants Auto-scaled based on your usage Scaling managed by you
Availability Kloudless manages high availability Kloudless manages high availability via Enterprise Clustering. Clustering support to enable high availability
Security Data encrypted in transit and at rest. 24/7 Ops team on call Same as “Cloud” except your data is isolated. Access is restricted to your IPs Inherits your security and regulatory compliance promises. No data passes through Kloudless infrastructure

The Level Up mini series is wrapping up soon –don’t miss it! If you can’t wait until our next blog post and want to start using Kloudless Enterprise today, drop us a line at hello@kloudless.com.

Level Up: A Developer’s Intro to Kloudless Enterprise (Part I of III)

Get all the benefits of Kloudless without using Kloudless servers.

That’s right. Now, you choose where your data goes. Introducing, the latest version of Kloudless: Kloudless Enterprise.

Kloudless Enterprise is a virtual appliance that is installed to private infrastructure. This means that you can host Kloudless in your own private cloud or on-premises data center.

Unlike the Cloud or Cloud Private Install versions of our service, the data exchanged between your application and storage services never passes through Kloudless’s infrastructure. Your servers communicate directly with the file storage services.

In the next two blog posts, we’ll cover how Kloudless Enterprise differs from our Cloud Private Install and the security/performance benefits of going on-premises.

On Premises Illustration
Stay tuned for Part II! Interested in hosting Kloudless Enterprise on your own servers? Shoot us an email at hello@kloudless.com.

VIP Kloud Treatment

If you built and deployed an application without contacting us, you are using our Cloud version, which is hosted on Amazon Web Services, multi-tenant, and fully managed by us. However, as the saying goes, “too many chefs in the kitchen can ruin the broth.” (This isn’t the best analogy, plus only we can ruin our broth, but you get the point.).

For a number of reasons (data security, special support requirements, etc.), not all developers want to be part of a multi-tenant facility. That’s why we’re introducing our new VIP service: the Cloud Private Install.

Cloud Private Installs are hosted on an instance in a service provider of your choice (AWS, Azure, Rackspace, etc.), with all features from the Cloud version supported. The installation is fully managed by Kloudless, including all infrastructure requirements, security updates and application upgrades. Based on the performance and availability you need, we can come up with the right instance category and bring up the virtual appliance in the appropriate regions of the cloud provider of your choosing.

Benefits of the Cloud Private Install include:

  • Superior networking performance and data throughput (e.g. Up to 10 Gbps on AWS)
  • Multi-tenant capable: CPU affinity can be optimized for either single or multiple tenants
  • Complementary proxy tool, enabling you to connect to on-premises storage repositories in your customer’s infrastructure
  • Custom SLAs and higher tiers of support

Our Cloud version and Cloud Private Install come with 99.9% uptime and a dedicated Ops team available 24/7, so regardless of which edition you’re using, you’re in good hands.

Interested in becoming a VIP? Get started with a Kloudless developer account and drop us a line at hello@kloudless.com.