Using Beam to Make AI Predictions with SageMaker

Introduction

In this guide, you will feed December 2022 electricity demand data in Tokyo (Tokyo Electric Power Grid Data) into XGBoost, creating a model that can predict the electricity demand from the area’s temperature and humidity. This model, which will be deployed on SageMaker, can receive data from IoT devices with Soracom Beam enabled, and make timely predictions powered by AI.

AI Prediction with SageMaker Architecture

More generally, any predictive model deployed on your AWS account’s SageMaker can be configured for use by Soracom’s AWS account through Soracom Beam. The authentication required to access the models on SageMaker can be offloaded away from your devices, since Beam will take care of the authentication and the devices only need Beam’s entry point.

Use the Oregon (us-west-2) AWS region for this service.
Amazon SageMaker has only confirmed SageMaker's stable operation in the Oregon (us-west-2) region. We found that some of the code on this page do not work as intended in other regions such as Tokyo (ap-northeast-1).

Step 1: Prepare the Data

You must first create an Amazon SageMaker notebook instance to prep the data accordingly.

Sign in to your AWS account and open Amazon SageMaker.
Change your region to Oregon (us-west-2).
Scroll down on the left nav bar and click Notebook instances, then click Create notebook instance on the top right.
Enter a Notebook instance name. (i.e. beam-sagemaker)
Click the IAM role drop-down menu and select Create a new role
Click the Any S3 bucket option, then click Create role.
Click Create notebook instance at the bottom of the page.
Wait until the status column of the notebook instance that you created reads In Service instead of Pending.

Click the reload button (left of the actions drop-down menu) to see any updates to your instance’s status.

Step 2: Use Jupyter Notebook to Create a Model

Using the notebook instance from Step 1, you will create an AI model that predicts energy consumption levels from the time of day, temperature, and humidity.

Create a Jupyter Notebook Instance

Confirm that the notebook instance that you plan on using is InService under the status column. Click the name of the instance you plan on using. (e.g., beam-sagemaker, this text should be in blue.)
Click Open Jupyter on the top right.
Click New then click conda_python3 This will create a Jupyter Notebook.

Create and Test the AI Model

Insert the code below into the cell and click Run.

# import libraries
import boto3, sagemaker
import uuid, urllib.request, os
import numpy as np
import pandas as pd

This will import all the necessary libraries.

If you are completely unfamiliar with Jupyter Notebook and would like to learn, refer to Jupyter Notebook's Official User Documentation.

In the next cell, paste the below code block and click Run

# Define IAM role
role = sagemaker.get_execution_role()
prefix = 'sagemaker/beam-xgboost'
my_region = boto3.session.Session().region_name
xgboost_container = sagemaker.image_uris.retrieve(framework="xgboost",region=my_region,version='1.5-1')
print("my_region: " + my_region + ", xgboost_container: " + xgboost_container)

This shows your region and XGBoost container URL.

my_region: us-west-2,
xgboost_container: 123456789012.dkr.ecr.us-west-2.amazonaws.com/sagemaker-xgboost:1.5-1

The latest container image for Sagemaker, with updated support for XGBoost can be found at Docker Registry Paths and Example Code XGBoost.

Paste the code below in the next cell and click Run.

In line 2, the S3 bucket name is declared. Change this as needed.

bucket_name = 'beam-sagemaker-' + str(uuid.uuid4())
print("bucket_name: " + bucket_name)
s3 = boto3.resource('s3')
try:
    if my_region == 'us-east-1':
        s3.create_bucket(Bucket=bucket_name)
    else:
        s3.create_bucket(Bucket=bucket_name, CreateBucketConfiguration={ 'LocationConstraint': my_region })
    print('S3 bucket created successfully')
except Exception as e:
    print('S3 error: ',e)

This creates a new Amazon S3 bucket.

bucket_name: beam-sagemaker-XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
S3 bucket created successfully