1. Managed Services: Doing more for our communities
12/11/2018
Dr Rhys Smith

2. Agenda A change in requirements, a change in direction
Managed Services
for our membership
for ourselves and our peers
Cloud orchestration
A tale of two approaches
What do we hope to gain from all this?
12/11/2018
Managed Services: Doing more for our communities

3. Jisc has traditionally run national services
A bit of background
Jisc has traditionally run national services
Let our membership connect themselves
Or let the market provide 3rd party help
E.g.
UK federation – 1,135 members, 2,304 entities
About 1/3 out-sourced, 2/3rds in-house
Eduroam - > 275 members
Almost all run in-house
Assent - ~ 20 members
All run in-house
12/11/2018
Managed Services: Doing more for our communities

4. A change in requirements
Membership heavily moving towards out-sourcing and cloud
Not a new thing, but seems to have accelerated in the last year or so
Market Failure
3rd party providers for UKf okay, but expensive
Not much for eduroam/assent, despite our nudges
12/11/2018
Managed Services: Doing more for our communities

5. Many members (esp. smaller) members:
Drivers
Many members (esp. smaller) members:
Cannot devote the time and effort for such specialised services
Are consequently running old/out of date software
Can’t take advantage of new features and initiatives
Alongside:
Continuing push to move to cloud and out-sourced provision of “non core” services
12/11/2018
Managed Services: Doing more for our communities

6. A change in direction
Many members asking “why can’t you do this for us?”
So we decided to do just that.
12/11/2018
Managed Services: Doing more for our communities

7. Managed Services for our Membership
12/11/2018
Managed Services: Doing more for our communities

8. Jisc Assent Jisc Liberate
Jisc Liberate - a set of cloud-based, managed, Access Management services
And a URL re-writing web proxy (no logo!)
Single management interface, shared configuration
Jisc Assent
12/11/2018
Managed Services: Doing more for our communities

9. Liberate As easy as… Subscribe Deploy VPN (optional)
Use web portal to configure connection to LDAP, configure services
Relax
12/11/2018
Managed Services: Doing more for our communities

10. Simple interface to manage:
Liberate Portal
Simple interface to manage:
LDAP connection
Service specific settings (e.g. SAML IdP consent, logos, MD, etc)
VPN connection
Log viewing
12/11/2018
Managed Services: Doing more for our communities

11. VPN back to home network
Secure by Design
VPN back to home network
Optional, but recommended
OpenVPN based
Client on the network, connects to server on the managed VMs
Download appliance or config
Appliance is a ~6MB OpenWrt VM (ova).
(If no VPN, we require LDAPS)
12/11/2018
Managed Services: Doing more for our communities

12. Partly customisable login page (more options soon)
Liberate Portal - SAML
Shibboleth v3 instance
Consent option
Partly customisable login page (more options soon)
Configure extra local MD
Configure attributes and attribute release
Currently support simple, scoped and mapped attributes. Scripted in the future.
12/11/2018
Managed Services: Doing more for our communities

13. Liberate Portal – (edu|gov)roam
All the basic stuff (incl. Upstream servers)
Manage:
Realms
VLANs
VLAN mappings
Map to a VLAN based on realm or LDAP group
APs and wireless controllers
12/11/2018
Managed Services: Doing more for our communities

14. Behind the scenes of Liberate
12/11/2018
Managed Services: Doing more for our communities

15. Portal interacts with AWS APIs
Custom Orchestration
Custom built portal
Portal interacts with AWS APIs
Each new service spins up an AWS VM from a custom Debian AMI
Portal pushes config to VMs via custom management daemon
Designed to be zero touch
Configuration gets pushed
Minor updates happen automatically
Major updates, we simply update the AMI and then rebuild
Elastic IP – no downtime
(well, almost… VPN needs to reconnect)
12/11/2018
Managed Services: Doing more for our communities

16. Responding to customer demand Increase take-up of services
Why are we doing this?
Responding to customer demand
Increase take-up of services
When it’s easy to deploy, more will join and use
Increase quality level of services
If we’re running stuff, we can keep it up to date and secure, and help customers implement things like Sirtfi & R&S
Reduce sector cost
Liberate has already reduced the cost of outsourcing as other providers have dropped prices after we launched
12/11/2018
Managed Services: Doing more for our communities

17. Managed Federation Service
12/11/2018
Managed Services: Doing more for our communities

18. So that’s managed services for users of the infrastructures
Up a level in the stack
So that’s managed services for users of the infrastructures
Can we apply the lessons and design patters of managed services to the infrastructures themselves?
We can… so we’re rebuilding the UKf to be a managed service for our own use
12/11/2018
Managed Services: Doing more for our communities

19. A bit of background (again)
– all infrastructure on Sun h/w and Solaris
2015- All rebuilt manually (to a deadline) on RHEL on Azure
2018 – Rebuilding again in a much more manageable way – docker based microservices
12/11/2018
Managed Services: Doing more for our communities

20. Splitting the federation into two
Frontend – distribution Infr
MD & MDQ distribution
CDS
Management portal
Test IdP/SP
WUGEN
etc
Backend – management Infr
Entity Repository
MD aggregate & MDQ creation
Orchestration
Signing
etc
12/11/2018
Managed Services: Doing more for our communities

21. Requirements for our Federation
Differing requirements:
Multiple backend systems
Gitlab, Jenkins, UK MDA, etc
High availability desirable but not essential, security important
Multiple Public facing systems
UKf Metadata distribution: ~10m requests / ~100TB traffic a year
CDS: ~ 40 million DS flows a year
Management portal
High availability CRITICAL - 100% uptime desired.
12/11/2018
Jisc Trust & Identity

22. Under the hood - Frontend
Global distribution platform built on Azure
Traffic manager fronting multiple (4+ initially) regions
Multiple docker hosts per region, load balancer in front
Individual docker nodes – no swarm, etc. Each host managed via docker-machine and docker-compose
Resiliency
Zero touch CI – build/test/release/deploy
IPv4 & IPv6 support
12/11/2018
Managed Services: Doing more for our communities

23. MFS Frontend – Global Footprint
Azure Traffic Manager
(Priority Routing)
Azure LB
Docker Node
Service
Azure LB
Docker Node
Service
Azure LB
Docker Node
Service
Azure LB
Docker Node
Service
Etc
Region A
Region B
Region C
Region D
12/11/2018
Jisc Trust & Identity

24. Under the hood - Backend
Docker engine swarm mode for backend
Resiliency within swarm for each service
cloudstor:azure driver for volume portability across nodes
Running
Gitlab
Jenkins
Management API (based on Shibboleth MDA)
Shibboleth MDA to create aggregates/MDQ
All backed up to AWS (hahaha)
12/11/2018
Managed Services: Doing more for our communities

25. MFS Backend – for each MFS instance
Azure Load Balancer
Docker Swarm
Docker node
Docker node
Docker node
Service
Service
Service
Azure region of choice
12/11/2018
Jisc Trust & Identity

26. UKf infrastructure is now > 20 RHEL VMs, all manually built.
Why are we doing this?
UKf infrastructure is now > 20 RHEL VMs, all manually built.
Want to make this much more modern and self-managing – reduce workload on support staff on server admin
Can concentrate on helping our members instead!
12/11/2018
Managed Services: Doing more for our communities

27. A side effect… Offering a Managed Federation Service
12/11/2018
Managed Services: Doing more for our communities

28. Spreading the joy So
We’ll have an infrastructure that we can spin up an instance of pretty quickly and easily, and that largely manages itself
Why not offer that service to others for their own use?
12/11/2018
Managed Services: Doing more for our communities

29. MFS offering - technical
Jisc MFS (name T.B.C.)
A collection of open source tools and orchestration routines that can run a federation the size of the UKf
Primarily aimed at helping smaller federations get world- class tooling quickly, cheaply, and easily
Letting them concentrate on interacting with their customers
(though bigger federations welcome also!)
Note: signing expected to be done locally in first instance
MFS user chooses local Azure DC
12/11/2018
Managed Services: Doing more for our communities

30. Non-technical aspects
Support
For federation operator only
Offering a set of managed tooling, not running the federation
Considering shared support with participants (follow the sun)
Currently piloting, about to start technical pilots
12/11/2018
Managed Services: Doing more for our communities

31. What do we gain from this?
We’re doing it anyway
Might as well offer to others at cost + a fairly small margin
Increase the quality of MD worldwide
Newer and smaller edugain participants often have problems
Steep learning curve!
Good, strict, tooling and a low cost might alleviate this
So we have to block fewer entities!
12/11/2018
Managed Services: Doing more for our communities

32. Things to chat to me about over a beer
Is everywhere else seeing this change in requirements?
How are you responding?
What areas can we collaborate in?
Want to play with Liberate?
Interested in reselling Liberate?
Interested in piloting/using our MFS?
12/11/2018
Managed Services: Doing more for our communities

33. Chief technical architect, trust & identity
Rhys Smith
Chief technical architect, trust & identity
12/11/2018
Managed Services: Doing more for our communities