As the CTO for a technology consulting services company and Professor of a cloud solutions course at the University of Houston the number one cloud question I get asked is, “Is the cloud secure?” Once I prove that the cloud is as secure as you need it to be the next question that gets asked is “where do I start?” Before I can give them any advice on where to get started they immediately start listing off 30 things they want to do.
For example, I was recently speaking to Manager of Infrastructure for a large health care provider where I was able to get him over the security hurdles he set up in his mind. Once he was confident that we could create a cloud environment in AWS that would be secure, elastic and durable he said, “I want to move all my database servers to the cloud asap.” I explained to him that the big bang approach rarely works and his stakeholders would most likely want to see some value first.
The rest of this post illustrates, in a overly simplistic way, how to show value quickly by going from data to insights in an hour using AWS RDS (MySQL and SQL Server Standard 2014), AWS Data Migration Service and PowerBI.
The Setup
Before we can go from data to insights in an hour we need to identify our data source. In this example we are using an on-premises SQL Server Standard 2014 edition (the source) and a AWS RDS MySQL instance (the target). Once the source and destination are identified open the AWS console and navigate to the database category and click on DMS. The next several steps will walk through what is needed to get the source data migrated to the destination.
Step 1: Create a replication instance using AWS Data Migration Services
Step 2: Connect source and destination endpoints.
The replication instance gets created and once completed test connectivity to the end points. In our case there is a SQL Server Standard Edition On-Premises and a MySql RDS Instance running. Why not Aurora? As of today (10/14/2016) PowerBI does not support Aurora but it does support MySQL. Why not SQL Server to SQL Server? That would be too easy and we need a cheaper alternative that provides enough bang for the buck. Ultimately the goal is focused on how quickly we can show value and not what end points should be used. If you get an error testing your connection check your security group as that will be the problem 99% of the time.
Step 3:Configure a task.
To keep everything simple just migrate everything from the source to the destination. For a complete breakdown of all the options available when creating a task click here.
Step 4: Run the newly created task.
Keep in mind this is a simplistic example of what you can do with AWS Data Migration Services click here for more information about what else is possible.
Step 5: After the task has completed review the results.
Notice that in the Table Statistics tab the Employees and Sales tables have been migrated.
Step 6: Query the MySQL instance just verification.
Use JackDB or some equivalent query tool to connect to MySQL. You can create a free trail and give it a go. Another reason to use JackDB is it allows you to run queries against Amazon RedShift (in addition to several other popular databases).
From Data to Insights
Now that the data has been migrated to an AWS MySQL RDS instance its time to produce some results. Amazon QuickSight could have been used to produce nice dashboard reports from our migrated data. However, QuickSight is still in preview. Nope for this part PowerBI is the best option (download PowerBI Desktop here). Once in the PowerBI environment Get Data and select MySQL data source. Lets finish up by adding PowerBI to the mix.
Step 7: Grab PowerBI desktop edition.
Once you have PowerBI select “Get Data” and select “MySQL” from the list. Enter Server and Database name (check your AWS console/RDS/Instances for your MySql Instance details). Be sure to include the entire endpoint location including the port i.e.
mysqldbinstance.cikpxzx0qopv.us-east-1.rds.amazonaws.com:3306
Step 8: Finally, once connected you will get your list of tables from MySQL. Select the tables and data you want and create whatever dashboards your heart desires.
All in all this took less than an hour to configure, run, and create.
Final Thoughts
Again this was a very simple sampling of what is possible. AWS DWS can be used to schedule the replication tasks. You can also move data to RedShift and still use PowerBI. The possibilities are endless but it is so much easier to show time to value by taking something simple like a data set, migrating it to the cloud and producing some tangible results quickly.
