AWS Research Workbench with Intelligent Automation using ServiceNow
With growing use of AWS Cloud across different industry segments for frictionless business, the use case of “Enabling Scientific Research” leveraging Cloud has unique benefits. Research is a very specialized field driven by a community of “Researchers” who want to focus on “Discovering Science than Servers”. Researchers day-to-day work requires processing data, collaborating online, and trying to maintain labs remotely. There is a need to democratize research computing so that everyone can use that easily.
Working closely with our AWS partners, Relevance Lab is creating an AWS “Research Workbench” powered by Intelligent Automation that can enable use of Cloud by Research Institutions and Researcher’s a frictionless manner.
Core functionality needed
- Basic need of High End and Research focused enterprises to be able to leverage AWS products seamlessly for research oriented business needs.
- Specialized roles – Principle Investigator, Researchers under one or many Research projects with different funding sources (Public and private).
- Ability to collaborate with Intramural and Extramural researchers.
- Specialized tools and software needs for an Analytics solution – AWS SageMaker, EMR, AI/ML, HPC, data security, secure Workspaces, large data sets sharing capability etc.
- Need for proper AWS Management & Governance with the ability to manage Self-Service (ITSM or custom portals) based lifecycle management (Provisioning, Managing, De-provisioning of users and assets).
- Proper cost and budget management and controls.
Additional challenges for Research Projects
- Massive Volumes of Data.
- Cross functional research teams.
- Research data management with compliance and security considerations.
- Leveraging new techniques of AI/ML, serverless computing, spot instances for HPC etc.
Scientific community has to adapt these challenges and AWS Cloud provides the platform for collaboration, on-demand resources and scale in a secure and compliant manner. Bringing together relevant AWS tools to create a bundle of Research Workbench makes this easier.
Catering to research needs special attention to the use-cases that may come up. For example a researcher may be working on a data science project using AWS Sagemaker notebooks and a large volume of research data in an S3 bucket. Given the sensitive nature of data, the access to the bucket may need to be secured within the organization and accessible only from within the specific network. Also a researcher may only need to access his own data and computing resources. We have developed a security model around the same which addresses such needs. The researchers can only access the resources from a Workspace created for them for that purpose.
To cater to the above the solution encompasses a “Research Portal” for user interactions and a specialized “Research Workbench” for collaborating on tools and data.
- Research Portal – Managed with existing ITSM Self Service Portals like ServiceNow.
- Research Workbench – Created by using AWS standard products, Service Catalog and Control Tower to enforce governance.
The above features allow creating and managing the lifecycle of a Research within an enterprise by leveraging investments in existing ITSM Portal and providing a seamless experience for AWS consumption. The solution leverages existing best practices of AWS Control services with Control Tower, Service Catalog, secure Access and automated provisioning/deprovisioning of resources. A critical part of such a Research Portal is proper cost management and tracking of research budgets and consumption against the same.
The following diagram explains the building blocks of a Research Workbench solution deployed with integration to ITSM Platforms like ServiceNow and using the AWS Service Management connector.
The reference deployment architecture using AWS Control Tower (CT) best practices is explained below. The access is controlled using AWS Simple AD and IAM roles.
The entire cycle of onboarding new researchers and provisioning assets for their research is automated using RLCatalyst BOTs solution with 1-Click deployment while still following the ITSM best practices as explained below.
Research Workbench Features
Following is a sample list of features planned (this is an indicative list only and not comprehensive)
Summary of Solution benefits
Based on the pre-built functionality of ServiceNow Self Service Portal, AWS standard products and our custom solutions are integrating the two platforms with a specialized research focussed use case. The following benefits includes:
- Quick start solution targeting Academic and Research Institutions – New and existing AWS customers.
- Existing customers with ITSM investments.
- Using existing ITSM platforms (ServiceNow, Jira Service Desk, Freshservice).
- Focusing on primarily “Built on AWS Solution” with standard products.
- AWS Control Tower, Service Catalog, ITSM Connector, Sagemaker, Workspaces, EC2, S3, RDS, EMR etc.
- Deployment options.
- Per customer Research Solution deployment (using customer Cloud and ITSM resources).
- Hosted solution offered to customers with (Managed Services based Cloud and ITSM platforms).
- RLCatalyst leveraged Solution(Automation, Service Portal, Observability and Cost Governance) add-ons.
- Pre-built solution to address 80-90% standard needs with scope of some customer specific customizations.
- Ability to on-board new customer in 3-4 weeks based on pre-built offering with agility and low onboarding costs.