1. Digital Services Contracting Professional Development MVP Program Release 4 Classroom Session
January 2017
Duration: 3 minutes
Timing: 8:00-8:03 am
Presenter: ICF/Peter Bonner
Welcome! Congratulations on making it through this very intense program!
It has been a busy few months, and this week we are wrapping things up so you can graduate! Woohoo!
As you may have noticed, there are tables with each of your group names on them, so please sit with your LDA groups for now.
These next few slides will go over some of the logistics for the classroom session.

2. Logistics Room is open from 7:30 a.m. to 4:30 p.m. every day
Internet access
Restrooms
Water, coffee, and kitchen
Duration: 2 minutes
Timing: 8:03-8:05 am
Presenter: ICF/Peter Bonner
Facilitator Notes:
Room is open from 7:30 am to 4:30 pm each day
Internet and password are on the white board.
During lunch, the USDS team will be available for optional brown bag sessions where further questions and discussions can occur.

3. Where You’ve Been
Release 1 – Digital Services in 21st Century Government
Understanding what digital services are, how digital services are designed and delivered, and the current role of a government digital services professional
A scan of the digital services market – potential sources of supply and how to communicate with them pre-solicitation
Release 2 – Understanding What You’re Buying
Exploring the need or outcome you’re trying to achieve with what you’re buying
Identifying and partnering with key stakeholders who will impact your acquisition
Conducting market research and preparing to develop your acquisition strategy
Duration: 2 minutes
Timing: 8:05-8:07 am
Presenter: ICF/Peter Bonner
Facilitator Notes:
Release 1 focused on understanding the landscape of digital services today.
Release 2 focused on gaining an understanding of what you’re buying by exploring the need or outcome you’re trying to achieve, identifying and partnering with key stakeholders who will impact the acquisition, and finally thinking about what the acquisition strategy will look like.

4. Release 4 – Awarding & Administering Digital Service Contracts
Where You’ve Been
Release 3 – How Do You Buy?
Utilizing techniques such as the lean canvas for acquisition planning
Analyzing evaluation criteria and the various elements of a procurement package
Identifying your organization’s readiness for agile and the role that plays in your acquisition strategy
Release 4 – Awarding & Administering Digital Service Contracts
Awarding digital service contracts by assessing the readiness of the technical evaluation team and implementing evaluation methods and criteria to evaluate vendor maturity
Managing digital services delivery
Identifying when failure occurs and what to do when it does
Duration: 2 minutes
Timing: 8:07-8:09 am
Presenter: ICF/Peter Bonner
Facilitator Notes:
In Release 3, focused on how to buy digital service solutions through development of your acquisition strategy and evaluation criteria, all in an effort to support your customer’s needs. You also learned about acquiring digital services– from identifying the role that security plays in digital service contracts, to defining evaluation criteria based on a given acquisition strategy.
In Release 4, you evaluated vendors who deliver digital services using instantaneous, objective metrics on project health, developed via appropriately applied lean thinking and agile development methods while experimenting with flexible contract design and administration strategies.
You explored awarding digital service contracts by assessing the readiness of the technical evaluation team, implementing evaluation methods and criteria to evaluate vendor maturity, etc.
You also learned about digital services delivery and how to identify software engineering practices for high-quality digital services, identify when failure actually occurs, and determine how to execute an exit strategy.

5. Agenda LUNCH LUNCH LUNCH Day 1 Day 2 Day 3 Morning Morning Morning
Welcome and Agenda
LDA/Shark Tank Prep
Shadowing/ Stakeholder Interview Debrief
LUNCH
Afternoon
MAP Scenario Conclusion
Guest Speaker
Leading Change
Morning
LDA/Shark Tank Final Prep
Guest Speaker – Digital Services Delivery
LUNCH
Afternoon
Shark Tank Pitches
Morning
Course Assessment
Program Feedback
DHS FLASH Team Panel
LUNCH
Afternoon
USDS “Big Picture” Discussion
GRADUATION!
Duration: 2 minutes
Timing: 8:09-8:11 am
Presented by: ICF/Peter Bonner
Facilitator Notes:
Day 1
Day 1 will be focusing on the week’s agenda and logistics. We will also give each group time to work on their shark tank pitch and ask questions of the USDS team as you get near a final state. Prior to lunch, each person will share with the cohort their shadowing and stakeholder experience, and the cohort will discuss these experiences and plans for moving forward at their own organization.
After lunch, we will work with the MAP Tech Tradeoff results and finish out the case study and have you select a vendor. You will then determine what negotiation tactics you willuse in getting the best price with the vendor, and do a role with invited guests from Acumen. At the conclusion, Acumen will discuss the vendor perspective on digital services delivery based on their experience with the whitehouse.gov project.
We will conclude today with a conversation about leading change at your organization.
Day 2
In Day 2, we will focus on finalizing your shark tank pitches so you are ready for the afternoon. We will have a guest speaker come in before lunch to talk about DevOps and performance monitoring in an agile environment. Then after lunch, it is Shark Tank time!
Day 3
In the morning, we will focus on completing the course assessment and gather overall program feedback. Then, we’ll hear more about innovative procurement strategies from the DHS team that recently awarded spots on its agency-wide agile software acquisition vehicle, which is called the Flexible Agile Support for the Homeland (or FLASH) vehicle. (Read more here:
After lunch, we’ll have some invited USDS guests who will talk to us about what’s next for digital services in government and how you fit in, followed by graduation and, of course, happy hour.

6. LDA/Shark Tank Prep Duration: 1 hour Timing: 8:16 – 9:30
Presented by: ICF/Peter Bonner

7. Shark Tank Prep Time
Please take this time to get into your groups and continue finalizing your “pitch” for tomorrow. You will have additional time tomorrow morning to finalize everything.
As a reminder:
You will pitch your product to a panel of faculty and guests.
The goal is to gain the panels’ commitment of time and resources to fully build out your product.
Presentation Rules
15 minutes to present; 10 minutes for questions
All team members must participate/talk in the presentation
Duration: 1 hour
Timing: 8:16 – 9:30
Presented by: ICF/Peter Bonner
Facilitator Notes:
Briefly explain how they will be judged during the shark tank game, and provide any pointers on what they should keep in mind.
In Release 3, you developed a plan to test your hypothesis, and you may have even started your test. For Iteration 4.A, you will complete your hypothesis test and compile your results.
You will NOT have to turn in a written summary to the program faculty. However, we recommend that you document the results of your test and how the results will affect the project moving forward (did they confirm your hypothesis, did they show where some details needed more attention, and so forth). This will be good preparation for your final presentation, where the test results are one of the items you will need to talk about.
Unlike the previous presentations, your final presentations will be a pitch in front of a panel comprised of USDS, OFPP or other guests.
The presentations will be similar to the reality show Shark Tank, where each group must pitch their Live Digital Assignment product to the panel to gain the panelists commitment of time and resources to pick up the project and build it out. See the Iteration 4.A Release 4 Live Digital Assignment Instructions link in the portal for additional details about what you need to include.
The presentation rules are:
Each group has 15 minutes to present and 10 minutes for questions
All team members must participate (meaning perform part of the presentation)
You can use from now until 9:30 am to work on your presentations. Traci and the rest of the facilitation team will be here to answer questions.

8. Live Digital Assignment Final Presentation
You will provide self-ratings and peer ratings (i.e., fellow team members)
Peer Rating Dimensions
Engaged Contribution
Collaborative Action
Rating panel (i.e., faculty, invited guests) will provide ratings
Panel Ratings Dimensions
Investment Buy-in
The Pitch Itself
NOTE: This slide and the next 4 slides were presented to students during the Iteration 4.A Iteration Planning Meeting. Briefly review each slide with the class to remind them of what they’re going to be evaluated on.
Facilitator Notes:
As you heard during the Course orientation, your live digital assignments will be graded on both self and peer ratings, as well as ratings from the panelists at the final presentations.
There are four rating dimensions, and they are weighted equally.
The self and peer ratings are: Engaged Contribution and Collaborative Action.
Because of the restructuring to the Live Digital Assignment, the panel ratings have change from what was in Orientation. The new ratings are: Investment Buy-in and the Pitch Itself.
The next few slides will show the details of each rating

9. Live Digital Assignment: Self Ratings & Peer Ratings
Engaged Contribution
Displays and applies adequate individual effort and tangible contributions to creating team work products, attends and actively participates in team events, motivates self and others to sustain enthusiasm and effective participation
Low
Medium
High
Does not display a “fire in the belly” or “quiet intensity of purpose,” participates at a low and/or inconsistent level, does not encourage fellow team members’ contributions
Tentative or intermittent willingness to “stretch” and apply his/her passion for the work, participates at a moderate level, inconsistent enthusiasm and support for fellow team members
Consistently displays passion for team excellence, strives to achieve “stretch” goals or overcome task challenges with a positive attitude, consistently participates at a high level, consistently inspires the best from fellow team members
Facilitator Notes:
Walk the students through the main definition of the Self/Peer rating and the different scores that can be earned.

10. Live Digital Assignment: Self Ratings & Peer Ratings
Collaborative Action
Contributes to a cohesive, inclusive and motivated team atmosphere, assesses relative team members’ areas of strength and works to combine efforts to create quality team products, identifies performance problems in a fact-based manner that facilitates mutual problem-solving, contributes to moving the team forward to achieving results
Low
Medium
High
At times, displays difficulty defining his/her role in the team, may engage in power dynamics (push-pull) when navigating team members’ contributions, and may not address performance problems constructively
Understands his/her role and authority dynamics within the team, has moderate understanding of how to combine/synthesize team members’ contributions, displays behaviors that include multiple perspectives, and acknowledges performance problems
Displays command of his/her role and useful ways of leveraging his/her expertise for the combined benefit of the group, regularly moves the team forward towards achieving results, maximizes the power of multiple perspectives within the team, and addresses performance problems in a fact- based manner to facilitate mutual problem-solving
Facilitator Notes:
Walk the students through the main definition of the Self/Peer rating and the different scores that can be earned.

11. Live Digital Assignment: Expert Panel Ratings
Investment Buy-In
The “Investors” will support this new endeavor because the information presented explains how the product could be implemented, how it will improve the end users experience in some way, and how the potential of the product indicates return on investment. No
Maybe
Yes
Is not able to illustrate the future of the product and what success would look like. Does not have attainable future goals or plans on how to implement or further test the product.
Has presented some of what success will look like, but the product’s future is either not fully thought out or has an unrealistic plan moving forward.
Successfully shows the panelist what success of the product will look like in the near and distant future. Has a good plan on how to move forward and implement or further test the product.
Facilitator Notes:
Walk the students through the main definition of the Panel rating and the different scores that can be earned.

12. Live Digital Assignment: Expert Panel Ratings
The Pitch Itself
The Presentation team was able to convey technical concepts, influencing techniques, and visuals to project in a compelling, engaging manner that inspires an investor to support the new endeavor. Builds trust and a bond with the investors such that enduring involvement is highly probable.
Needs Work OK
Great
Does not use the presentation in a way that influences a panelist to support their product. Introduces skepticism instead of building trust with the investor. May have relied more heavily on some team members to “carry” the presentation than a united message.
Presents the material well but leaves the panelist with some small doubts about the product. Covered the required elements but left the investors wanting more information.
Presentation was coherent, polished, and utilized all team members to gain the investors’ interest and trust in the product.
Facilitator Notes:
Walk the students through the main definition of the Panel rating and the different scores that can be earned.

13. Stakeholder Interview & Shadowing Debrief
Duration: 1 hour
Timing: 9:30-10:30
Presented by: Peter Bonner/Will Randolph

14. Stakeholder Interviews & Shadowing
Let’s review these activities:
Stakeholder Interviews: Helped you engage influencers and gatekeepers in your agency
Shadowing helped you:
Build your familiarity with modern design and development approaches.
Allow you to engage directly with digital services teams to learn about their work.
Build a network with digital services experts that you can leverage.
Duration: 5 minutes
Timing: 9:30-9:35
Presented by: Peter Bonner/Will Randolph
Facilitator Notes:
Summary of the Stakeholder Interview Assignment:
In this activity, you identified at least two and up to four individuals within your agency you believe are major influencers or gatekeepers in the digital services acquisition process and developed a strategy for engaging with them. You were asked to research these individuals and develop hypotheses that you could test during your conversation. Then, you developed out your conversation purpose and structure, practiced the conversation, and then completed it.
We’ll review the questions you were asked to reflect on on the following slides.
Summary of Shadowing Requirement: 
During the program, you were required to shadow a digital services implementation/delivery team, i.e., a team that is “on the ground” developing digital services. 
Purpose: Action shadowing will provide you with the opportunity to observe and interact with a digital services team in action as a “day in the life” experience. The action shadowing experience should have helped:
Build your familiarity with modern design and development approaches.
Allow you to engage directly with digital services teams to learn about their work.
Build a network with digital services experts that you can leverage.
Shadowing Opportunity Selection Criteria: You may choose to shadow a product manager, developer, designer, or anyone on a digital services delivery team within or outside of government. Examples of shadowing opportunities you might select include shadowing an agile team at GSA’s 18F, USDS, or within an agency and attending backlog grooming and sprint planning sessions with the team. 
Extra Participation Badge: Applied Skills badges will be provided to any student who completes more than one shadowing opportunity or more than the 16 hours required on the first shadowing opportunity. For example: One participant shadows two different groups for 16 hours each, and another participant shadows one group for 32 hours. Both of these participants would be eligible for an Applied Skills badge. 

15. So How Did Stakeholder Interviewing Go?
Provide a brief overview of your experience. Ensure you address the following:
Who did you interview? Why did you select these individuals?
What was the hypothesis you aimed to test during each interview? What was the result?
How influenced was each interviewee by your insights or ideas? How was this evidenced?
Is there anything you would change that would have made the conversation more productive? Were you able to have them "put down" any previous baggage they might have associated with the acquisition process?
How does this conversation impact your future conversations with this stakeholder?
What is your plan for further engagement with this stakeholder?
Duration: 50 minutes
Timing: 9:35-10:25
Presented by: Peter Bonner/Will Randolph
Facilitator Notes:
Briefly go over the instructions below. If you want to give them a few minutes to organize their thoughts around the questions on the slide, that is fine too.
Note that the questions students will be asked to answer are the same as the ones that were included in the portal description of this activity. They do not have to answer every question, the important thing is that they share their experience and discuss as a class.
You will listen to each person’s report out and encourage the students to comment or ask questions about their experience. You should also ask questions about the experience. On the slide are the general questions we want to use to spark conversation.
Structure:
Individual brief out
Group discussion
Instructions:
The presenter will:
Share your overall experience with your stakeholder interviews
Be sure to address each of the questions on the following slide
The audience will:
Ask additional questions about the experience; focus on areas that differed from your own experience
Time: Give each student a few minutes to share, you have 1 hour to get through all of the cohort for just the Stakeholder piece. The Shadowing piece is after the break.

16. Morning Break Duration: 10 minutes Timing: 10:25-10:35
Presented by: ASI Will Randolph
Facilitator Notes:
Instruct participants to take a 10 minute break.
Direct to restrooms and water cooler.

17. So How Did Shadowing Go?
Provide a brief overview of your experience. Ensure you address the following:
What shadowing opportunity did you identify?
What kind of activities occurred?
What was the person/group’s “superpower”?
What did you learn?
Did you observe or discuss best practices with the person/group you shadowed? Did you have any other key takeaways?
Duration: 55 minutes
Timing: 10:35-11:30
Presented by: Peter Bonner/Will Randolph
Facilitator Notes:
You will listen to each person’s report out and encourage the students to comment or ask questions about their experience. You should also ask questions about the experience. On the slide are the general questions we want to use to spark conversation.
Structure:
Individual brief out
Group discussion
Instructions:
The presenter will:
Share your overall experience with Shadowing
Be sure to address each of the questions on the following slide
The audience will:
Ask additional questions about the experience; focus on areas that differed from your own experience
Time: Give each student a few minutes to share, you have 1 hour to get through all of the cohort for the Shadowing piece

18. Lunch
The lunch break will run from 11:30-12:30pm.

19. MAP Scenario Conclusion
Duration: 1 hour
Timing: 12:30-1:30
Presented by: Peter Bonner

20. MAP Case Study Overview
Over the course of the program, you supported MAP by doing the following:
Develop a Product Vision
Conduct Market Research
Develop an Acquisition Strategy using the Lean Canvas
Develop Evaluation Criteria
Make a Source Selection
How did the last step go?
Duration: 5 minutes
Timing: 12:30-12:35
Presented by: Peter Bonner
Facilitator Notes:
Conduct a brief review of the various steps they took/executed with the MAP Case Study.

21. The Technical Tradeoff
In Iteration 4.A, you were given offers by two vendors: Always Shippin’ and Semper Agile. You were asked to:
Evaluate each vendor offer
Identify the items each offeror can make to both enhance their offer in consideration of the identified weaknesses and to make a tradeoff decision
Complete the Evaluation Worksheet
Choose a vendor!
Duration: 5 minutes
Timing: 12:35-12:40
Presented by: Peter Bonner
Facilitator Notes:
Refresh their memory on what they did in Iteration 4.A.
GOAL: The objective of this activity is to arrive at the best value solution for the MAP case study. The agency has initially budgeted $500,000 for the initial MVP effort, however there is more funding available. Your assignment is to identify which company represents the best value based on program priorities for this effort, consistent with the evaluation plan and RFP, and the tradeoffs that can be made by reviewing the technical evaluation and price submitted. 
Review the entire activity here so you can be ready to facilitate the debate activity:
Once you go through the slide, ask participants to split into two groups based on who they selected as their vendor. Designate areas of the room for each group to gather.
Who did you choose as your vendor?

22. Activity Part 1 – Discuss Your Tradeoffs
Discuss the tradeoffs considered in selecting your vendor – 10 minutes
Choose a presenter from your group to discuss your tradeoffs with the class – 5 minute debrief per group
Duration: 20 minutes
Timing: 12:40-1:00
Presented by: Peter Bonner
Facilitator Notes:
Part 1 – Discuss Your Tradeoffs
Once the groups have split into two, explain the activity to participants. Each group will:
Discuss the tradeoff worksheet they completed as individuals during Release 4. Work together in their groups to combine their individual analysis into one group worksheet highlighting the tradeoffs that led them to that vendor; they should do this using the white board, or a large sheet of paper on the wall; give 10 minutes for this.
Once they are done, have them choose a presenter to walk the class through their selection and rationale; give 10 minutes for this.

23. Activity Part 2 – Prepping for the Negotiation
Negotiation objective: To come to an agreement on the terms of the contract so the government can make the final award and begin the work.
In your groups, discuss what key terms you want to negotiate and what you want the result to be in that negotiation.
Duration: 25 minutes
Timing: 1:00-1:25
Presented by: Peter Bonner
Facilitator Notes:
Part 2 – Prepping for the Negotiation:
After the Part 1 briefings, have them get back into their groups and determine additional questions to ask vendors if discussions were opened up.
Also determine a negotiation strategy for getting the best value to prepare for the mock negotiation with Acumen (our guest vendor).
This part should take you until 1:25, then move to the next slide to get the groups set up in preparation for Acumen’s arrival and mock negotiations.
Remind them of the below as they work through this part.
Negotiations:
Objective of the Negotiation:
To come to an agreement on the terms of the contract so the government can make the final award and begin the work. In your teams, discuss what key terms you want to negotiate and what you want the result to be in that negotiation. Both the vendor and government sides should work up a list of questions for the other side and establish your posture during the negotiation of each area. Also determine areas where you cannot be flexible; i.e. date of final deliverable, price, etc.
In digital services, remember to address these main areas at a minimum:
Terms and Conditions
Service Agreements
Government Standards
Privacy
Intellectual Property
Risk Management
Here are the strategies they learned about in Iteration 4.A. Remind them of this and then ask them to consider the above questions.
Use a Team Approach. Each member should have a clear understanding of:
The requirement and the solicitation
The source selection processes and evaluation factors/criteria outlined in the solicitation
The Independent Cost Estimate/Independent Government Cost Estimate, how it was prepared, and the budgeted amount available
The negotiation objectives for the specific negotiation session
Questions to think about:
What is your negotiation tactic? Team approach, win-win, good cop bad cop?
Who are the key members of your team?
Where should it take place?
What are your deal breakers?
If they have internet access, ask them to pull up their portal and look at the 4A course on Negotiations for some tactics, do-don’ts, etc.:

24. Activity Part 3 – Let’s Make a Deal!
Welcome to our guests from Acumen!
Remember the negotiation strategy you just came up with? It is time to try it out.
Each group should select 2-3 members to conduct the negotiation; the remaining group members should observe and take notes
You will have about 20 minutes to negotiate, and then a 10 minute retrospective
Duration: 35 minutes
Timing: 1:25-2:00
Presented by: Peter Bonner/Acumen
Facilitator Notes:
Each group will negotiate with Acumen concurrently; i.e. Semper Agile group will have one Acumen person to negotiate with, while Always Shippin’ does the same. Have each group select 2-3 people to conduct the negotiation, while the rest of the group should observe and take notes, but not interfere. Give participants 5 minutes to get organized and select who will play the various roles within the negotiation. Then, assign a role player from Acumen to each group. Then, have the negotiations within each group last for 20 minutes. Rotate around the room as these negotiations are going on to answer questions and/or otherwise facilitate discussion.
For the last 10 minutes, have each team share their experience.
Here are some Probing Questions after the negotiation is over:
   1.  Did you meet your objectives?
   2.  Did you use any of the do’s in your online learning, or any don’t’s?
   3.  Did the lead negotiator have an agenda and follow it?
   4.  Did you summarize areas of agreement as you talked?
   5.  What about facial expressions or body language; did it tell you anything or lead you to change your approach?

25. Afternoon Break 1 Duration: 5 minutes Timing: 2:00-2:05
Presented by: ICF/Peter Bonner
Facilitator Notes:
Instruct participants to take a quick 5 minute break while we get things set up for our guest speakers.
Direct to restrooms and water cooler.

26. Guest Speaker: Acumen Duration: 55 minutes Timing: 2:05-3:00
Presented by: ICF/Peter Bonner
Facilitator Notes:
Peter will introduce our Acumen guest speakers who will be discussing the vendor perspective working on whitehouse.gov project. Acumen will speak to digital services delivery and the contract administration perspective on the vendor side, with Traci providing CO input based on her experience working with Acumen in this capacity.

27. Afternoon Break 2 Duration: 10 minutes Timing: 3:00-3:10
Presented by: Traci
Facilitator Notes:
Instruct participants to take a 10 minute break.
Direct to restrooms and water cooler.

28. Leading Change Duration: 40 minutes Timing: 3:10-4:50
Presented by: Traci
Facilitator Notes:

29. Placeholder for Traci Duration: 40 minutes Timing: 3:10-4:50
Presented by: Traci
Facilitator Notes:

30. Graduation Details Duration: 5 minutes Timing: 3:50-3:55
Presented by: Traci

31. Graduation Day!
That’s right, you are graduating in two days. Here is the run down:
Duration: 5 minutes
Timing: 3:50-3:55
Presented by: Traci

32. Preview of Tomorrow Duration: 5 minutes Timing: 3:55-4:00 pm
Facilitator: ICF/Peter Bonner

33. Day 2 Agenda LDA/Shark Tank Final Prep Morning
Guest Speaker – Digital Services Delivery
Lunch (12:00-1:00 pm)
Afternoon
Shark Tank Pitches
Duration: 5 minutes
Timing: 3:55-4:00 pm
Facilitator: ICF/Peter Bonner
Facilitator Notes: Refresh participants on what’s coming up tomorrow and remind them that they’ll have a bit more time to prep their shark tank pitches before delivering them in the afternoon.