” … Microsoft released its Azure Active Directory Sync Services tool on Monday… Azure AD Sync Services is notable for being Microsoft’s intended replacement tool for the Directory Synchronization (DirSync) tool. Both tools are used to synchronize (or copy) user identities managed through Active Directory in organizations. …”
For further details on the ESI Suite and all of its components click here
Click here to download a copy of ESI 3.5
EMC Microsoft Community site | https://www.emc.com/everythingmicrosoft
Excellent ESI overview videos by David Lloyd (@davlloyd) | http://www.youtube.com/user/davlloyd69 | http://youtu.be/9effRPY7uLU | http://youtu.be/CveZDDBf6pE | http://youtu.be/VVdOA7LsZw0 | http://youtu.be/0Vy3uNlAbsc | http://youtu.be/_22OiKxsV_Y
ESI and VNX | http://youtu.be/rL3o9hG72HI
EMC Storage Integrator (ESI) for Windows is the ultimate set of tools to ease storage provisioning for the Windows administrator. In addition, ESI will provision application level components (such as Exchange databases, SharePoint sites, and SQL databases). With ESI, the Windows admin will no longer have to wait for other teams to perform tasks on their behalf in order to provision storage for their applications.
The Windows admin will use familiar tools (such as MMC, PowerShell, and System Center) to work with storage. ESI combines the functionality of many desperate tools into one GUI interface and one set of PowerShell scripts. This will ease the learning curve and streamline the process for systems management.
Example process improvement:
Should you be a PowerShell scripter, ESI includes a rich set of PowerShell libraries for provisioning and managing of EMC storage.
In addition, the ESI suite includes full integration of EMC products into System Center (SCOM, SCVMM, and SCO)
Storage monitoring with ESI Management Packs (SCOM)
Storage management from within Virtual Machine Manager (SCVMM)
ESI System Center Orchestrator Integration Pack
EMC Storage Integrator (ESI) for Windows Suite is a set of tools for Microsoft Windows and Microsoft applications administrators. ESI for Windows provides the ability to view, provision, and manage block and file storage for Microsoft Windows, Exchange, SQL, and SharePoint.
The suite includes:
- ESI for Windows and ESI PowerShell toolkit
- Storage System adaptors
- ESI hypervisor support
- ESI adaptors for: SharePoint, Exchange, SQL, Appsync, RecoverPoint
- ESI SCOM Management Packs
- EMC Hyper-V VSS requestor
EMC Storage Integrator (ESI) for Windows has a UI based on Microsoft Management Console (MMC). ESI can be run as a stand-alone tool or as part of an MMC snap-in.
ESI supports storage provisioning and discovery for Windows virtual machines (VMs) running on Hyper-V, Citrix XenServer, and VMWare vSphere and vCenter.
- Create VHD and VHDX files, and pass-through SCSI disks
- Create hosts disks and cluster shared volumes (CSVs)
- Create VMDK files and raw device mapping (RDM) disks with or without virtual compatibility mode
- Create SCSI disks
- View datastores
- Create VHD files and storage repositories
ESI supports the EMC Symmetrix® VMAX® series, EMC VNX® series, EMC VNXe® series, and EMC CLARiiON® CX® fourth generation (CX4) series of storage systems. ESI requires that you install the applicable adapter for your specific storage systems.
Summary of the key New Product Features included in this release (3.5):
- Support for Microsoft SQL Server
- Integrated Microsoft Active Directory role based access (RBAC)
- Support for RecoverPoint and/or Appsync
- Support for Linux servers
- VPLEX Health Monitoring
This release adds the following new functionality:
- ESI SQL Server Adaptor to view and manage SQL Server databases and provision storage for SQL Server
- ESI VPLEX Adaptor to monitor health of VPLEX systems in SCOM
- ESI AppSync Adaptor to automate application protection with EMC AppSync
- Use Microsoft Active Directory Domain Services (AD DS) or Active Directory Lightweight Directory Services (AD LDS) with ESI to persist connection settings in a central location for all supported applications and systems
- ESI Linux Adaptor to manage Linux hosts
- ESI VNXe Adaptor to manage LUN groups and unified snapshots for EMC VNXe 3200 systems
- ESI RecoverPoint Adaptor to manage EMC RecoverPoint Bookmarks
- ESI Exchange Integration to update HA Extension with the ESI GUI for easy DAG and database failover operations
Storage System and hypervisor software prerequisites
My notes from the session OneDrive for Business
75% of businesses allow personal devices, Only 25% have a BYOD policy
Productivity versus Compliance
File/Sync/Share – Very similar to SyncPlicity
All my files in one place
1TB of storage
Sync client – various clients
Office client integration <- Cool feature
Co-authoring <- This is really cool. Being to see other authors updates on a document in realtime
Search & Discovery
Part of SharePoint and O365
File upload size 2GB
Post to Yammer
“Shared with me” view
Which version should I use?
SharePoint features update every 3 years
O365 updates very frequently
Custom apps can be developed on top of OneDrive
Site Folders – Team site document libraries
Recent Documents – across all office clients that you use <- Cool feature.
Launch “new” documents from O365
Rich preview from O365 <- Cool feature.
Edit from O365 with co-authoring in real time ( you can see the other person edits live) <- Awesome feature.
External sharing is optional via policy
Post to Yammer – can limit to groups and users
Search – my docs and shared with me
Workflows, Check in/out, workflows – just like in SharePoint
Email with attachment or link to doc with Outlook in O365
Code name Oslo “Office Graph” – New Search and Discovery across all Office content, includes edits, comments, trending, liked, relationships, adheres to permissions (only shows content that I have access to)
Mac Sync Client
Native Android app
Encryption at Rest
Oslo + Office Graph
New Admin panel for SharePoint on-premise
Pricing – Standalone offering $2.50/user/month, 1TB per user, includes Office Online
Part of Office365 ProPlus
My notes from the TechEd session on SQL inside of Azure VMs
This stuff is confusing. No standard storage terminology being used.
Someone asked if the disks in the BLOBS where Thick or Thin. Answer: We don’t use those terms.
Performance Guidance for SQL Server in Windows Azure Virtual Machines
Azure VMs tie to Azure Storage – need to understand how Azure Storage works
Azure service is a shared service, multi-tenet service
Azure BLOB storage
VHDs stored in BLOB
3 copies automatic – persistent copies
Shared service – noisy neighbors can be an issue
No direct performance correlation between disk requests made by VM and the underlying physical disk
No performance SLA’s
500TB / BLOB
Total requests rate 20,000 /sec
Geo-redundant storage = 10gb/sec Ingress, 20 gb/sec Egress
Local redundant = 20gb/sec Ingress 30 gb/sec Egress
Two tiers of service for VMs
VM Basic tier = A0-A4 tiers
VM Standard tier = A5-A9 tiers
C:- 127GB – persistent
D: Temp drive – 20GB-605GB depend on tier – may be wiped on reboot – local disk, low latency
Data Disks – up to 1TB each (thin)
Where and when you provision your VM matters – the service can move your VM around
Maintenance operations have an effect on performance – notified from Microsoft when they do maintenance
C:- D:are local disks versus Azure disks stored in a BLOB
– need to test to determine scenarios for performance of where to put what
VM Disk Cache
“Host Cache Preference” when creating a disk
Two tier cache:
Tier 1 – Most recently accessed data stored in RAM of physical virtualization host. This cache is sharedby all VMs on that host
Tier 2 – Less recently accessed data stored on the local disk of the physical host (disk based cache). Ther is cache space reserved to each VM OS disk and Data disk based VM size.
Reduce transaction against Azure Storage
Reduce disk IO latency when IO rate is low
Impr0ve boot time
Network Bandwidth and Delay
I/O go through separate stack from Guest VM network
“Cloud Service” is a management, config, security, networking and service model boundary
Affinity group : Compute and storage to be placed close to each other
Going through load balancers adds latency
IO paths on Azure VM
Use Data disks (not local OS disk) for DBs larger than 10GB
Use “None” for high OLTP – random I/O
Use “Read” if workload is sensitive to latency
Use “Read/Write” for sporadic disk access – not a write back cache
Static data, seldom accessed
Cache setting can be changed via PowerShell
VM instance size and limits
Place user DB & logs on different data disks
If on OS drive:
Avoid OS disk stripping, use SQL striping (ie. file group)
Warm up disk before measuring performance
After initialize the file by writing some bit patterns sequentially using SQLIO
Consider enabling SQL data compression (row, page) for I/O bound workloads
CPU usage can increase
Enable instant file initialization
Don’t use auto-grow, pre-allocate instead
Use 64KB DAU
Placement of data files
Option 1: DB across multiple files with a file group
Option 2: Create an “OS volume” on top of multiple disk
Option 3: Multiple striped volumes each with one or more data disks (based on specific I/O performance requirements)(Host level RAID 0)
Place on OS or Data disks
Don’t use D: temp drive – performance issues can occur
Disk warm up
Disk Load balancers – cold disks get less performance, hot disks get better performance
Idol disks (cold) for 20 mins will reduce your performance. Azure remove disk load balancers from cold disks.
Manually Warm up disks for 10 mins – will get you better performance
Use a Script to warm up
Monitor tab for storage account
Storage analytics logging
Minimal or Verbose
Aggregated data place into hidden place in BLOB “&log” – accessible via PowerShell
Moving to the cloud doesn’t solve app design issues
Performance baseline needed prior to moving to Azure in order to understand performance
- Backup directly to BLOB storage via URL – within the same data center, if another datacenter will be charged egress
- Backup to regular disk
Repurposing of data -??
How to populate test/dev?
You may think that EMC and Microsoft are bitter competitors, and you may be right because in some areas they do fiercely compete. But more often than not, the two companies work together to solve customer problems with joint solutions from both sides. Even with competing products, overlapping functionality, and differences in approach, it is the customer who ultimately drives the decision. That decision is rarely a one sided architecture. Most often we see a blend of technologies from both EMC and Microsoft that come together to solve business issues and initiatives.
Microsoft has competitive technologies, guidance, and solutions in several areas. EMC has a different set of direction and federation partners.
Such examples are:
- Windows Azure public cloud services – These services compete directly against VMware vCHS public cloud services.
- Microsoft’s Azure started off as PaaS (Platform as a Service) with providing custom .NET application hosting and SQL database service. Azure then added many more IaaS (Infrastructure as a Service) services to their offering. These are such as: Virtual Machine hosting, Backup, AD, Storage, Web sites, and a Service Bus. Azure is making inroads into bridging on premise with public cloud, but the transition is yet seamless.
- VMware’s vCHS (vCloud Hybrid Services) is primarily a IaaS service providing seamless transition from on premise to public cloud. VMware is slowly adding PaaS service for custom application hosting.
- VMware also offers the customer the choice of service providers, whereas Microsoft only offers Azure through their own hosting facility.
- Microsoft has virtualization technologies that are based on Windows Server, Hyper-V and System Center. These technologies compete directly against our federation story and solutions with VMware.
- Microsoft is advising customers to steer away from SANs and intelligent arrays, and towards DAS (Direct Attached Storage). This guidance is based on the fact that Microsoft has added high availability into their applications (Exchange and SQL) and intelligent storage services into their software (Storage Spaces and ReFS).
Microsoft however is hedging its bets by providing technology into their software that will leverage and take advantage of shared intelligent storage.
- ODX (Offloaded Data Transfer) – This provides the offloading of storage functions to array and takes the load off of the server.
- SMB 3.0 – This provides a robust file transfers protocol for CIFS and allows for critical workloads to leverage NAS.
EMC and Microsoft have a long standing and mutually beneficial relationship. This relationship is formalized as EMC being a Microsoft “Global Alliance Partner”. Together EMC and Microsoft provide Industry leadership in cloud computing and IT Transformation.
There are several areas that show the commitment to this partnership:
- EMC has engineers and hardware in lab at Microsoft’s headquarters in Redmond, WA. In addition, EMC has hardware and staff at the Microsoft Technology Centers (MTC) located in various cities across the US and globally. The MTC’s are used in customer engagements for Envisioning, Architectural Design, and POC’s
- EMC and Microsoft have a Cooperative Support Agreement that facilitates technical support between the companies on behalf of the customer. The customer will need to open a service ticket with both tech support organizations, and then the two companies will work together to resolve the customer’s issues.
- EMC actively integrates Microsoft technologies into EMC technologies, and typically releases the solutions first to market. EMC has over 300 published technical solutions and another 100 data sheets on Microsoft technologies.
- EMC offers VSPEX Proven Infrastructures for the Microsoft stack, including: Virtualized Exchange, Virtualized SQL, Virtualized SharePoint, and Microsoft Private Cloud (converged infrastructure for Hyper-V and System Center).
- EMC Consulting has a very large global consulting practice for Microsoft technologies.
- EMC has created a Microsoft centric community site that is open to partners and customers. This site contains whitepapers, EMC Proven Solutions for Microsoft, videos, case studies, and open discussion threads on EMC/Microsoft solutions (www.emc.com/everythingmicrosoft).
As a Microsoft Certified Partner (with 16 competencies and a 21-time Partner of the Year winner), EMC offers our joint customers expertise and technologies to confidently integrate their Microsoft applications with EMC solutions. EMC and Microsoft have programs and solutions that can accelerate customers to transform their IT to private clouds, and provide IT as a Service. EMC is committed to the Microsoft platform and ecosystem, and actively continues to add value by added functionality, solutions, and leadership.
Even though the two companies compete in several areas, every customer of EMC has Microsoft in their environments to some degree. It is imperative that we talk to our customers about their Microsoft applications and communicate EMC’s value for Microsoft.
Recap of my session at Microsoft TechEd 2014 – Microsoft Azure Storage
Azure Storage is a cloud based storage offering which is hosted in 16 regions across the world.
- An object store for unstructured data
- REST API access only
- Flat namespace
Up to 60 MB/sec throughput
Up to 1TB/BLOB
Charged by storage actual consumption
- Network mounted virtual disk (inside of Azure for Azure-based VMs)
- Stored in BLOB
- up to 1TB Disk
- Max 8KB IOPS – 500 IOPS
- Snapshot and copy
- Up to 60MB/sec throughput
- Disks are exclusive to a single VM
- noSQL cloud store
- Odata protocol (AtomPub or JSON)
- Reliable messaging system
- Low latency, high throughput
5> Files (new offering as of May 2014)
- Azure File Share – SMB and REST
- SMB 2.1
- Up to 5TB File Shares
- Up to 1TB file
- SMB 2.1, access only available within region, otherwise use REST
- REST accessible from within Azure and from Internet
- True directory structure
- Up to 60 MB/sec throughput per share
- Charged based file size
- Can’t boot VM’s off of Azure File
- Max 8KB IOPS – 1000 IOPS
Azure Storage Protection options:
LRS – Local Redundant Storage –
3 copies within a zone (facility)
ZRS – Zone Redundant Storage (for block BLOBs only)
3 copies between zones but within the same Region
GRS – 6 copies of data across two regions (3 in each region) – Async consistent
Read-only access to GRS
Read-only access to secondary data even if primary in unavailable
Strongly consistent within the primary region
Eventually consistent across regions
All writes go to primary
Move Tbs of data into and out of Azure BLOBS by shipping disks
Encrypted with BitLocker
Azure Copy for files
Utility to copy files into Azure Files (via REST API)
Tools and libraries:
.NET, Java, C++
Zone redundant storage for BLOBs