Download presentation
Presentation is loading. Please wait.
Published byLucas Hill Modified over 8 years ago
1
WINDOWS AZURE PLATFORM DEEP DIVE: FABRIC, ROLES AND OTHER FUN Luka Debeljak CEE Windows Azure Lead http://blogs.msdn.com/lukad
3
AGENDA Cloud Computing and Windows Azure Platform Windows Azure Fabric Windows Azure Roles (Web, Worker, VMRole) Deployment, Upgrade, Failover Security Remote Access, Admin access, full IIS Windows Azure AppFabric Access Control Caching Connecting Private and Public Clouds Windows Azure Connect Windows Azure Service Bus Summary
4
CLOUD COMPUTING, A DEFINITION… Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model promotes availability and is composed of five essential characteristics, three service models, and four deployment models. -National Institute of Standards and Technology http://csrc.nist.gov/groups/SNS/cloud-computing/index.html
5
THE ELECTRICAL GRID
6
CLOUD SERVICES Packaged Software Storage Servers Networking O/S Middleware Virtualization Data Applications Runtime You manage Infrastructure (as a Service) Storage Servers Networking O/S Middleware Virtualization Data Applications Runtime Managed by vendor You manage Platform (as a Service) Managed by vendor You manage Storage Servers Networking O/S Middleware Virtualization Applications Runtime Data Software (as a Service) Managed by vendor Storage Servers Networking O/S Middleware Virtualization Applications Runtime Data
7
WHAT IS THE WINDOWS AZURE PLATFORM? The Windows Azure platform is an internet-scale cloud computing services platform hosted in Microsoft data centers. Provides a range of functionality to build applications that span from consumer Web to enterprise scenarios. The Windows Azure platform includes: Windows Azure SQL Azure Windows Azure AppFabric
8
WINDOWS AZURE PLATFORM Scalable compute and storage Automated service management Familiar tools, technologies, languages Relational storage for the cloud Consistent development model Automated database management Connect existing apps to the cloud Connect through network boundaries Easily control authorization to apps
9
BASIC FUNCTIONALITY Storage Compute Developer SDK
10
DEPLOYING A SERVICE MANUALLY Resource allocation Machines must be chosen to host roles of the service Fault domains, update domains, resource utilization, hosting environment, etc. Procure additional hardware if necessary IP addresses must be acquired Provisioning Machines must be setup Virtual machines created Applications configured DNS setup Load balancers must be programmed Upgrades Locate appropriate machines Update the software/settings as necessary Only bring down a subset of the service at a time Maintaining service health Software faults must be handled Hardware failures will occur Logging infrastructure is provided to diagnose issues This is ongoing work…you’re never done
11
THE FABRIC CONTROLLER The Fabric Controller: Provisions machines Configures hardware Deploys applications Monitors the data center Manages everything
12
Switches WINDOWS AZURE FABRIC CONTROLLER Highly-available Fabric Controller Out-of-band communication – hardware control In-band communication – software control WS08 Hypervisor VM Control VM Service Roles Control Agent WS08 Node can be a VM or a physical machine Load-balancers
13
WINDOWS AZURE AUTOMATION Fabric Controller (FC) Maps declarative service specifications to available resources Manages service life cycle starting from bare metal Maintains system health and satisfies SLA What’s special about it Model-driven service management Enables utility-model shared fabric Automates hardware management “What” is needed Make it happen Fabric Switches Load-balancers Fabric Controller
14
LIFECYCLE OF A WINDOWS AZURE SERVICE Resource allocation Nodes are chosen based on constraints encoded in the service model Fault domains, update domains, resource utilization, hosting environment, etc. VIPs/LBs are reserved for each external interface described in the model Provisioning Allocated hardware is assigned a new goal state FC drives hardware into goal state Upgrades FC can upgrade a running service Maintaining service health Software faults must be handled Hardware failures will occur Logging infrastructure is provided to diagnose issues
15
WINDOWS AZURE SERVICE LIFECYCLE GOAL IS TO AUTOMATE LIFE CYCLE AS MUCH AS POSSIBLE Coding & Modeling New services and updates Provisioning Desired configuration Deployment Mapping and deploying to actual hardware Network configuration Maintain goal state Monitor React to events Automated Developer Developer/ Deployer
16
Load Balancer Channel Endpoint Interface Directory Resource MODELING SERVICES Fundamental Services Load Balancer Template automatically maps to service model Public Internet Background Process Role Background Process Role Front-end Web Role Front-end Web Role
17
CONFIGURATION Service Configuration ServiceDefinition.csdef – service model ServiceConfiguration.cscfg – instance data RoleEnvironment.GetConfigurationSettingValue() Don’t use web.config for values you wish to change at runtime Web.config change requires re-deploy
18
DEMO WINDOWS AZURE SOLUTION
19
DYNAMIC CONFIGURATION SETTINGS Purpose: Communicate settings to service roles There is no “registry” for services Application configuration settings Declared by developer Set by deployer System configuration settings Pre-declared, same kinds for all roles Instance ID, fault domain ID, update domain ID Assigned by the system In both cases, settings accessible at run time Via call-backs when values change
20
FAULT/UPDATE DOMAINS Allows you to specify what portion of your service can be offline at a time Fault domains are based on the topology of the data center Switch failure Statistical in nature Update domains are determined by what percentage of your service you will take out at a time for an upgrade You may experience outages for both at the same time System considers fault domains when allocating service roles Example: Don’t put all roles in same rack System considers update domains when upgrading a service Allocation is across fault domains Fault domains
21
THE FC KEEPS YOUR SERVICE RUNNING Windows Azure FC monitors the health of roles FC detects if a role dies A role can indicate it is unhealthy Upon learning a role is unhealthy Current state of the node is updated appropriately State machine kicks in again to drive us back into goals state Windows Azure FC monitors the health of the host If the node goes offline, FC will try to recover it If a failed node can’t be recovered, FC migrates role instances to a new node A suitable replacement location is found Existing role instances are notified of the configuration change
22
HOW UPGRADES ARE HANDLED FC can upgrade a running service Resources deployed to all nodes in parallel Done by updating one “update domain” at a time Update domains are logical and don’t need to be tied to a fault domain Goal state for a given node is updated when the appropriate update domain is reached Two modes of operation Manual Automatic Rollbacks are achieved with the same basic mechanism
23
UPGRADING YOUR APPLICATION Two Models: VIP Swap and In-Place Upgrade VIP Swap: Uses Staging and Production environments. Allows to quickly swap environments. Production: v1 Staging: v2, after swap then Production: v2 Staging: v1. In-Place Upgrade Performs a rolling upgrade on live service. Entire service or a single role Manual or Automatic across update domains
24
SERVICE ISOLATION AND SECURITY Your services are isolated from other services Can access resources declared in model only Local node resources – temp storage Network end-points Isolation using multiple mechanisms Automatic application of windows security patches Rolling operating system image upgrades Managed code Restriction of privileges Firewall Virtual Machine IP filtering
25
DATA SECURITY HYPERVISOR & VM SANDBOX Customer code run on dedicated virtual machines (VMs) VMs isolated by a Hyper-V based hypervisor All access to network and disk is mediated by a “root” virtual machine Network/Disk Hypervisor 1, 2, 4 or 8 CPUs, up to 14GB of memory Stripped down, hardened version of Windows Server 2008 Three virtual hard disks Limited number of device drivers Network connectivity restricted using host firewall Hyper-V based hypervisor
26
WINDOWS AZURE FC IS HIGHLY AVAILABLE FC is a cluster of 5-7 replicas Replicated state with automatic failover New primary picks up seamlessly from failed replica Even if all FC replicas are down, services continue to function Rolling upgrade support of FC itself FC cluster is modeled and controlled by a utility “root” FC Client Node FC Agent FC Core Object Model Disk Uncommitted Committed FC Core Object Model Disk Committed FC Core Object Model Disk Committed Replication system Primary FC NodeSecondary FC Node
27
WINDOWS AZURE FABRIC IS HIGHLY AVAILABLE Network has redundancy built in Redundant switches, load balancers, and access routers Services are deployed across fault domains Load balancers route traffic to active nodes only Windows Azure FC state check-pointed periodically Can roll-back to previous checkpoints Guards against corrupted FC state, loss of all replicated state, operator errors FC state is stored on multiple replicas across fault domains
28
SERVICE MONITORING SDK component providing distributed monitoring & data collection for cloud apps Support Standard Diagnostics APIs Trace, Debug normally Manage multiple role instances centrally Scalable Choose what to collect & when to collect it Event Logs, Trace/Debug, Performance Counters, IIS Logs, Crash Dumps, Arbitrary log files
29
WINDOWS AZURE DIAGNOSTICS Role Instance Starts Diagnostic Monitor Starts Monitor is configured Imperatively at Start time Remotely any time Monitor buffers data locally User can set a quota (FIFO) User initiates transfer to storage Scheduled or On Demand Role Role Instance Diagnostic Monitor Local directory storage
30
REMOTE DIAGNOSTICS ADMINISTRATION Modify diagnostics configuration from outside Azure Poll Interval Role Role Instance Diagnostic Monitor Local directory storage
31
WINDOWS AZURE PLATFORM MANAGEMENT PORTAL Faster and more user friendly portal for Windows Azure Platform services Enables more visibility and control Supports multiple users and roles Enable Remote Desktop and easily connect to Windows Azure instances Built on the Windows Azure Service Management APIs
32
DEMO WINDOWS AZURE MANAGEMENT & DIAGNOSTICS
33
ANATOMY OF A WINDOWS AZURE SERVICE A Windows Azure service consists of An isolation boundary A set of component roles, each with endpoints Numbered, multiple instances of each role A Windows Azure application behaves correctly when any role instance fails. All of this is specified in a service model Worker Role Web Role Web Role VM Role
34
COMPARISON OF ROLE TYPES Web Role : The role is hosted on IIS Worker Role : The role is an executable (you can create your own web server, host a database, …) VM Role : The role is the VM. Use Windows services, scheduled tasks, etc. You configure and maintain the OS. Control Abstraction (i.e. Less IT & Less Plumbing Code) Admin Web / Worker Role VM Role Web / Worker Role
35
VIRTUAL MACHINE ROLE VM Role provides additional control and flexibility on the Windows Azure environment, and makes it easy to run existing Windows applications on Windows Azure. Developers have full control over the OS image Ability to upload your own customized WS08R2 Enterprise images Operators can reboot, reimage and Remote Desktop Continue to benefit from automated service management
36
VM ROLE LIFECYCLE CloudOn-Premises Blob Storage Boot VHD Customize VHD Save Diff.VHD Base.VHD Identical/similar deployment instances using common uploaded OS image (base.VHD + diff.VHD) Additional Software Windows Azure Integration Components Agent Runtime Interface (topo, config, shutdown notification, …) Remote Desktop configurator Diagnostics Windows Azure Drives driver Generalize (Recommended)
37
CREATING A SERVICE: SERVICE DEFINITION
38
REMOTE DESKTOP Portal-integrated Remote Desktop access to instances within your deployment Use standard Remote Desktop client Support for all three role types – Web, worker, and VM Dynamic configuration of Remote Desktop settings Credentials Enable/Disable Expiration of credentials Ability to take instances on/off the load balancer for debugging
39
REMOTE DESKTOP UNDER THE HOOD Web Role with Remote Forwarder VIP Load Balancer Worker Role
40
REMOTE DESKTOP CONFIGURATION
41
REMOTE DESKTOP CONFIGURATION
43
SERVICE MODEL ENHANCEMENTS Windows Server 2008 R2 & IIS 7.5 Web Role is revamped to use full IIS Admin Access: Full administrative access to your role instances Network Modeling Well Known Ports: Easier to bring your existing applications to Azure Port Ranges: for “inbound traffic” (as opposed to 5 in the past) Load Balancer Control: Take instance on/off the load balancer for debugging purposes Network Filters: Enable traffic filtering between roles within a service Improved Automation Startup Tasks: Includes scripts that automate the preparation and configuration of roles Role Plugins: Ability to easily include “plug-ins” in the service definition (e.g., Built-in: Remote Desktop, virtual network, diagnostics; Samples: IIS Dev Mode, and more)
44
ADMIN ACCESS & STARTUP TASKS Scenarios Perform short, unattended setups on role startup. E.g. COM Component Registry Key Configuring a Windows Server Role, such as IIS Debugging Reminders Durability: Runtime state on OS image does not survive OS upgrade and hardware failure Drift: Service should behave correctly when any role instance fails
45
ADMIN ACCESS & STARTUP TASKS <Task commandline="relative\path\ToSetupExecutable" executionContext="limited|elevated" taskType="simple|foreground|background"/> Task Types: Simple [Default] – System waits for the task to exit before any other tasks are launched Background – System does not wait for the task to exit Foreground – Similar to background, except role is not restarted until all foreground tasks exit
46
FULL IIS – OVERVIEW Web Role will now use full IIS by default One Web role can run many Web sites Multiple sites, applications, virtual directories, and bindings can be modeled Everything else can be scripted (using admin access where necessary)
47
FULL IIS SCENARIO - WALKTHROUGH PhaseKey Steps Local DevelopmentUse Visual Studio solution to code & deploy to Compute Emulator Edit local content files without redeploying to Compute Emulator First-time Publish (~10 Min)Deploy service to Windows Azure Ensure Windows Azure service definition (csdef/cscfg) configures Web role: To enable remote WMSVC, WebDeploy, PowerShell, etc. As single instance Iterative Development (seconds) Continue coding in same Visual Studio solution Push intermittent changes (e.g. w/ Web Deploy) Deployment of Final Version (~10 Min) Deploy service to Windows Azure Note: If service is now multi-instance, Remote Desktop can be used, but other remote mgmt. tools will get load- balanced to arbitrary instances. Mitigation: Run mgmt. tools inside Remote Desktop session
48
DEMO WINDOWS AZURE REMOTE DESKTOP
49
AGENDA Cloud Computing and Windows Azure Platform Windows Azure Fabric Windows Azure Roles (Web, Worker, VMRole) Deployment, Upgrade, Failover Security Remote Access, Admin access, full IIS Windows Azure AppFabric Access Control Caching Connecting Private and Public Clouds Windows Azure Connect Windows Azure Service Bus Summary
50
WINDOWS AZURE PLATFORM APPFABRIC Infrastructure services for the cloud Goal is to enhance the Azure platform offering with solutions to common application requirements Address typical Internet scenarios Make it standards-based for interoperability Make it easy if you already know.NET
51
APPFABRIC – IN A NUTSHELL AppFabric Services Caching Service Bus Access Control Integration Composite App (WF, WCF).NET + AppFabric Composition Model & Tools AppFabric Container Multi-tenant, high-density, scaled-out infrastructure Composition runtime AppFabric Management
52
APPFABRIC AND INTEROP AppFabric SDK for Java AppFabric SDK for Ruby AppFabric SDK for PHP
53
ACCESS CONTROL Makes it easy to authenticate and authorize users Integration Single Sign On and centralized authorization into your web applications Standards-based identity providers Enterprise directories (e.g. Active Directory Federation Server v2.0) Web identities (e.g. Windows Live ID, Google, Yahoo!, and Facebook)
54
A PRIMER ON CLAIMS ?
55
Trust Read Policy Claims Processing Application Logic AuthenticateGet Token Send Token (claims) Establish Session Relying Party Identity Provider Security Token Service CLAIMS BASED ACCESS
56
Application Server MICROSOFT IDENTITY PLATFORM ADFS 2.0 End User 5. Send claims Establish relationship using federation metadata trust Identity Provider 1. Read policy 2. Read policy 4. Get claims 3. Authenticate Relying Party Windows Identity Foundation Windows Identity Foundation App Business Logic App Business Logic Windows CardSpace 2.0 Windows CardSpace 2.0
57
FEDERATION trust Relying Party User Application 2. Redirect to STS 1. Attempt access 3. Home realm discovery 4. Redirect to STS 5. Authenticate Windows Identity Foundation AD FS 2.0 Company A Company B
58
trust Relying Party User Application 9. Post claims 7. Post claims 8. Get claims 6. Get claims Company A Company B FEDERATION Windows Identity Foundation
59
DEMO WINDOWS AZURE ACCESS CONTROL
60
WHAT IS CACHING? An explicit, distributed, in-memory application cache for all kinds of data (CLR objects, rows, XML, Binary data etc.) Fuse "memory" across machines into a unified cache Unified Cache View Caching clients can be across machines or processes Clients Access the Cache as if it was a large single cache Cache Layer distributes data across the various cache nodes
61
APPFABRIC CACHING A distributed, in-memory cache for applications running in Windows Azure In-memory cache located near your Windows Azure applications Simple administration Based off the proven Windows Server AppFabric Caching capabilities Benefits Highly scalable 64-bit caching solution with low latency and high throughput Can dynamically increase and decrease as needed End-user doesn’t have to bother with configuration, deployment, or management of their cache
62
CACHING FEATURES Capabilities ASP.NET providers for session state and page output caching Extreme low latency with the local cache Caches any managed object (CLR objects, rows, XML, Binary Data…) No object size limits No serialization costs for local caching Easily integrates into existing applications Same managed interfaces as Windows Server AppFabric Caching Secured by the Access Control Service
63
USING CACHE THROUGH CODE Simple Access APIs //create DataCacheFactory based on config file var dcf = new DataCacheFactory(); //get the cache named "TestCache" var cache = dcf.GetCache("TestCache"); //Add an item called "Test" - throws if exists cache.Add("Test", new Data {TheData = "Test" }); //Put an item called "Test2" - overwrites if exists cache.Put("Test2", new Data { TheData = "Test2" }); //Get "Test3" - add if not in cache (cache-aside) var test3 = cache.Get("Test3") as Data;
64
ASP.NET INTEGRATION (NO CODE NEEDED) Only simple config change needed
65
APPFABRIC CACHING SCENARIOS Caching Reference Data (e.g. product catalog) A version of the authoritative data, refreshed periodically Large number of accesses, mostly read Caching Activity-Oriented Data (e.g. shopping cart, session state) Data generated as part of the app activity, typically logged back to a backend database Needs read, write access Caching Resource-Oriented Data (flight inventory) Authoritative data, modified by transactions Needs read, write access
66
CACHING OFFERS Size based offers, discrete increments in sizes 128 MB 256 MB … and more! Can upgrade/downgrade between the above sizes
67
DEMO WINDOWS AZURE CACHING
68
AGENDA Cloud Computing and Windows Azure Platform Windows Azure Fabric Windows Azure Roles (Web, Worker, VMRole) Deployment, Upgrade, Failover Security Remote Access, Admin access, full IIS Windows Azure AppFabric Access Control Caching Connecting Private and Public Clouds Windows Azure Connect Windows Azure Service Bus Summary
69
WINDOWS AZURE APPFABRIC CONNECTIVITY 69 Application #1Application #2 Direct Connection facilitated by Service Bus if that is best connection mechanism. Text XML Graphics Binary Data Streaming Firewall Send Receive Exchange messages between loosely coupled, composite applications. Send Receive
70
MULTICAST PUBLISH/SUBSCRIBE Builds on the relay and direct connect connectivity capabilities Sender Receivers
71
RICH SET OF CONNECTIVITY BINDINGS
72
SERVICE BUS FUTURES Orchestration Routing Coordination Transformation Orchestration Routing Coordination Transformation Rich, declarative content-based routing, document transformation, and process coordination. Service Management Naming, Discovery Monitoring Service Management Naming, Discovery Monitoring ? Consistent, modeling- friendly management surface and service virtualization capabilities Messaging Queuing Pub/Sub Reliable Transfer Messaging Queuing Pub/Sub Reliable Transfer Reliable, transaction- aware cloud messaging infrastructure for business apps. Connectivity Service Relay Protocol Tunnel Eventing, Push Connectivity Service Relay Protocol Tunnel Eventing, Push Rich options for interconnecting apps across network boundaries
73
WINDOWS AZURE CONNECT IN CONTEXT CLOUDENTERPRISE Data Synchronization SQL Azure Data Sync Application-layer Connectivity & Messaging Service Bus Security Federated Identity and Access Contro l Secure Network Connectivity Windows Azure Connect
74
WINDOWS AZURE CONNECT Secure network connectivity between on-premises and cloud Supports standard IP protocols Enables hybrid apps access to on-premises servers Allows remote administration of Windows Azure apps Simple setup and management Integrated with WA Service Model Web, Worker and VM Roles supported Enterprise Windows Azure
75
WINDOWS AZURE CONNECT CLOSER LOOK Enable WA Roles for external connectivity via service model Enable external computers for connectivity by installing Connect agent Win Server 2008, 2008 R2, Vista, and Win7 supported platforms Network policy managed through WA portal Granular control over connectivity Automatic setup of secure IPv6 network between connected role instances and external computers Tunnel firewalls/NAT’s through hosted SSL-based relay service Secured via end-to-end IPSec DNS name resolution Role ARole B Role C (multiple VM’s) Windows Azure Enterprise Dev machines Databases Relay
76
WINDOWS AZURE SERVICE DEPLOYMENT To use Connect with a WA service, enable one or more of its Roles For Web & Worker Role, include the Connect plug-in as part of Service Model (.csdef file) For VM role, install the Connect agent in VHD image using the Connect VM install package Connect agent will automatically be deployed for each new role instance that starts up Connect agent configuration managed through the ServiceConfiguration (.cscfg) file One required setting - “ActivationToken” Unique per-subscription token, accessed from Admin UI
77
ON-PREMISES DEPLOYMENT Local computers are enabled for connectivity by installing & activating the Connect agent Web-based installation link Retrieved from admin UI Contains per-subscription activation token embedded in URL Standalone install package Reads activation token from registry key Enables installation using existing S/W distribution tools Connect agent tray icon & client UI View activation state & connectivity status Refresh network policy Connect agent automatically manages network connectivity Sets up virtual network adapter “Auto-connects” to Connect relay service as needed Configures IPSec policy based on network policy Enables DNS name resolution Automatically syncs latest network policies
78
MANAGEMENT OF NETWORK POLICY Connect network policy managed through Windows Azure admin portal Managed on a per-subscription basis Local computers are organized into Groups E.g. “SQL Servers”, “My Laptops”, “Project Foo” A computer can only belong to a single group at a time Newly activated computers are ‘unassigned’ by default WA Roles can be connected to Groups Enables network connectivity between all Role instances (VM’s) and local computers in the Group WA Connect does not control connectivity between Roles or Role instances (done through existing mechanisms) Groups can be connected to other Groups Enables network connectivity between computers in each group In addition, a Group can be ‘interconnected’ - enables connectivity within a group Useful for ad-hoc & roaming scenarios
79
CONNECT NETWORK POLICY - EXAMPLE SERVER1SERVER2 Windows Azure SERVER3 DEV_LAPTOP1 Role A Instance3 Instance2 Instance Role B Instance3 Instance2 Instance DEV_LAPTOP2 My ServersMy Laptops
80
CONNECT AND DOMAIN-JOIN Connect plug-in supports domain-join of WA Roles to on- premises Active Directory Process to enable: Install Connect agent on DC / DNS server(s) For multiple DC environment, recommend creating dedicated Site Configure Connect plug-in to automatically join WA role instances to AD Specify credentials used for domain-join operation Specify target OU for WA role instances Specify list of domain users / groups to add to local Administrators group Configure network policy to enable connectivity between WA roles and DC / DNS servers New WA role instances will automatically be domain-joined Be aware: domain-joined WA Role instance != On-premises computer Role instance not guaranteed to persist local state; role instance identities may change over time General guidance – Role instances use AD identities vs. actively managed as a domain-joined computer
81
WINDOWS AZURE CONNECT - SCENARIOS WA Role accessing on-premise SQL server Or file server, line-of-business app, etc. Domain-join scenarios Control access to WA Role instances using domain accounts Web role using IIS Windows Integrated Auth Run role under domain account to access on- premises resources (e.g. SQL server secured with Windows Integrated Auth) Remote Powershell to WA Role instances Or remotely access a file share, event log, etc.
82
DEMO WINDOWS AZURE CONNECT
83
SUMMARY Cloud Computing and Windows Azure Platform Windows Azure Fabric Windows Azure Roles (Web, Worker, VMRole) Deployment, Upgrade, Failover Security Remote Access, Admin access, full IIS Windows Azure AppFabric Access Control Caching Connecting Private and Public Clouds Windows Azure Connect Windows Azure Service Bus
84
BENEFITS OF MOVING TO WINDOWS AZURE Transcend Complexity Focus IT on driving the business not maintenance Utilize familiar development tools Transparent ROI Agile Infrastructure Rapid deployment of new business solutions Scale capacity up and down on demand Sustained Advantage Low ongoing infrastructure costs Helps provide a secure and compliant platform Opportunity for extra functionality
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.