Azure Storage Services

Azure Storage Services

  1. Blob Storage
    1. Block blobs – Ordinary files up to 4.7 TB.
    2. Page blobs – Random access file up to 8 TB in size. Mostly VHD files that back VMs.
    3. Append blobs – like block blobs, but optimized for append operations.

 

  1. File Storage – Network file share that multiple VMs can share.
  2. Queue Storage – Store and retrieve messages.

 

Azure Storage Account

  1. General Purpose
    1. Standard (Magnetic disks)
    2. Premium (SSD disks, recommended for VMs)
  2. Blob Storage (Block blobs & Append blobs)

 

Replication

  1. Locally Redundant Storage (LRS: 3 copies in single data center)
  2. Zone Redundant Storage (ZRS)
  3. Geo Redundant Storage (GRS)
  4. Read Access – Geo Redundant Storage (RA – GRS)

 

+++++++++++++

Created resource group as it is required before creating storage account.

Created container inside blob service.

Created SQL Databases. Connected it using SSMS.

 

 

++++++++++

Tutorial 1

Create a azure data factory pipeline to read input file and transform the data using HDInsight on demand cluster and output the file. Input & Output file are used Azure Blob storage.

 

Steps –

  1. Setup storage account
  2. Create a blob service and upload files to that service. You may want to create specific folders.
  3. Create data factory.
  4. Create a linked service to connect azure data store and deploy.
  5. Create input & output data set.
  6. Create pipeline

 

 

Tutorial 2

Create a azure data factory pipeline to copy input file data to SQL database. Input & Output are Azure Blob storage & Azure SQL.

 

Copy Activity & Copy Activity Wizard

 

Tutorial 3

Read more about dataScience