Lean Networking An experiment in the application of best-in-class business practices to network and IT management

A Lean Perspective Lean is a compilation of “world class business practices” including The Toyota Production System, Waste Elimination Process/Continuous Improvement Other quality certifications (six sigma, ISO 900x)

Traditional Lean is immediately identifiable by these buzzwords: Just In Time ( ジャストインタイム ) (JIT) Just In Time Kanban ( 看板, also かんばん ) (engl.: Sign, Index Card) Kanban Muda ( 無駄, also ムダ ) (engl.: Waste) Muda Heijunka ( 平準化 ) (engl.: Production Smoothing) Heijunka Andon ( アンドン ) (engl.: Signboard) Andon Poka-yoke ( ポカヨケ ) (engl.: fail-safing - to avoid (yokeru) inadvertent errors (poka)) Poka-yoke Jidoka ( 自働化 ) (engl.: Autonomation - automation with human intelligence) Jidoka Kaizen ( 改善 ) (engl.: Continuous Improvement) Kaizen

Lean Purpose To eliminate all waste or non value added activities from a process. This is not a “once over” audit but a real- time activity Not meant to eliminate people, but to use them most wisely

The Essential Lean Tools have the following outcomes Identifies and eliminate waste quickly and efficiently Increase communication at all levels of the organization Reduce costs, improve quality and delivery in a safe environment Begin improvements immediately and empower workers to make improvements themselves

A time for Lean at Oakland One of Oakland’s Trustees and benefactors endows the Pawley Institute - a lean methodology think tank and training center Recent Fiscal pressures in the region and state requires rethinking funding strategies at even the Unit level. The expectation of network operations (24x7) with a limited staff (8x5) requires new efficiencies The right mix of people – the team synergy is open to trying new approaches to daily network operations

So … how to get Lean There are several “dialects”, cultural variations, para-lean, and lean-alikes. The textbook on Lean is written by Jeffery Liker (U of M), “The Toyota Way” (2003)

14 Principles from Jeffrey Liker’s “The Toyota Way” Base your Management decisions on a long-term philosophy, even at the expense of short term financial goals Create continuous process flow to bring problems to the surface Use “pull” systems to avoid over-production

14 Principles Continued Principle 4 Level out the Load (Heijunka) Reduce Muri – the overburden of people and equipment that results in safety, quality, and performance problems (vs. sustainable pace) Reduce Mura – or unevenness. If the workload or the type of work fluctuates wildly from one moment to the other, people (and machines) never "get into the rhythm" and waste time switching between tasks. The production stalls and restarts all the time.

14 Principles Continued Principle 5-7 Build a culture of stopping to fix problems, to get quality right the first time. Standardized tasks: these are the foundation for continuous improvement and employee empowerment Use visual control so no problems are hidden

14 Principles Continued Principle 8-10 Use only reliable, thoroughly tested technology that serves your people and process Grow leaders who thoroughly understand the work, live the philosophy and teach it to others Develop exceptional people and teams who follow your company's philosophy

14 Principles Continued Principle Respect your extended network of partners and suppliers by challenging them and helping them improve Go and see for yourself to thoroughly understand the situation (Genchi Genbutsu) Make decisions slowly by consensus, thoroughly considering all options; implement decisions rapidly

14 Principles Continued Principle 14 Become a learning organization through relentless reflection (Hansei) and continuous improvement (Kaizen)

The Experiment: Can lean principles be applied to IT management? These principles were originally designed for manufacturing but they have been ported to other areas of business and technology, including software and systems development, so … Will the outcomes actually result in the positive gains realized in other verticals: less waste, increased communication, increased quality and cost savings, and effective use of people? Network vertical provides a case study

Principle 1 - Application Base your Management principles on your long term philosophy All information technology initiatives are related to the University Mission, the strategic goals, and the OU in 2010 initiative. Every network component has a strategic orientation as it’s engine (projects are not simply industry driven or a person or divisional project)

Principle 2 - Application Create Continuous Process Flow to bring problems to the surface (there is also a concept of “design for rapid changeover”) Projects and upgrades happen cyclically (no “all at once” or “forklift” if possible … this way problems can be corrected on the way Change occurs during regularly scheduled weekly change management windows

Principle 3 - Application Use "Pull" systems to avoid overproduction The network is not “overbuilt”. Equipment is provided in an as-needed basis, with a small margin for growth. The shelf time for inventory is not more than 90 days (with the exception of spares) There is no active storage space … equipment is immediately moved to staging and then to production Redundancy/HA is minimized except where absolutely necessary or required

Principle 4 - Application Level out the load (Heijunka) Cyclical replacement keeps the network updated and evergreen on an ongoing basis. Age failures are mitigated Systems are designed to operate at the mid-point or just above of their processing capacity. Staff learns nuances of the technology over the cycle time (reduces staff training burnout)

Principle 5 - Application Build a culture of stopping to fix problems, to get quality right the first time The right solution is encouraged as a design option (and not simply the lowest cost or fastest implemented). Staff are encouraged to reveal and correct problems through self audit Management is friendly is asking the question “how do we do it right” first instead of “how fast can it be done, how expensive will it be”

Before and After After years of “The “get it done” method The stop to do it right method

Before and After

Principle 6 - Application Standardized tasks are the foundation for continuous improvement and employee empowerment LEAN works because each closet has standards. Parts are interchangeable, cables and cable lengths are standard. Staff are empowered to make changes during the pre-scheduled change periods Each staff member is an expert in their vertical product line

Principle 7 - Application Use visual control so no problems are hidden The five steps in the process: 1. Sort through and sort out 2. Set things in order and set limits 3. Shine and inspect through cleaning 4. Create and set standards 5. Educate and communicate

Step 1 Sort Through and Sort Out Legacy equipment in closets that have no owner, no purpose, and no power supply (hey … this thing isn’t even plugged in) is eliminated Asset inventory of every element in every closet is taken This increases morale and responsibility by ensuring that every area is ordered and is an element of pride.

Before and After

Step 2 Set things in order and set limits Everything in the network closets have their place. Every network closet is designed according to a common standard or layout Missing items and many problems can be solved visually Example: Patch cable color is used to identify the generation of networking and the category of the connection. In a previous iteration, patch cable boot color was used to identify length of cable.

Order and Limits

Step 3 Shine and inspect There is no closet or workspace that is not visited on a regular basis Cleaning and pickup is part of every visit (even if for a cross connect) Annually, the whole team walkthrough of the entire facility to evaluate, reorganize, and cleans

Another example … The last working day of December is a departmentally mandated “clean-up, clean out” day. This steps through each of the previous elements.

Step 4 – Create and Set Standards The team has worked to standardize at every “layer”. The standards are practical and pragmatic (clear descriptions are encouraged and the goal is to make it counter-intuitive to do a task or project in any other way) As an example, our wiring standards moved from a 100+ page document to a 22 page document.

Step 5 – Educate And Communicate Standards are not internal documents, they are linked to University Policy and then made available to the community Standards and policies communicated specifically to the distributed/divisional technology through a monthly roundtable meeting and via an Annual Retreat sponsored by Central IT. Annually, the central IT group holds campus awareness events where standards and policies are communicated to the whole campus External groups are made aware of the standards

Principle 8 - Application Use only reliable, thoroughly tested technology that serves your people and process Oakland has determined that we don’t want to be leading or trailing edge. “Best” technologies are evaluated from a variety of different perspectives: trade recognition, peer use and review, RFP with references

Principle 9 - Application Grow leaders who thoroughly understand the work, live the philosophy and teach it to others Network management is not about the next wire that is plugged in, it is about how we accomplish the goal. The CIO is committed to this fact

Principle 10 - Application Develop exceptional people and teams who follow your company's philosophy Training is a given in the Oakland Environment. Each staff member is given training stretch goals annually. Teams and teamwork are promoted above the individual

Principle 11- Application Respect your extended network of partners and suppliers by challenging them and helping them improve Projects at Oakland are opportunities for vendors to excel at what they do. To offer the best designs, the best products, and the best prices

Principle 12 - Application Go and see for yourself to thoroughly understand the situation (Genchi Genbutsu) Technical staff is encouraged not to manage by phone. Site visits are encouraged. Vendors are encouraged to visit the site. Annually, Sr. Management (The Division VP takes the unit management on a walk-around through the office and operational spaces.

Principle 13 - Application Make decisions slowly by consensus, thoroughly considering all options; implement decisions rapidly All perspectives are considered and consulted: customer, (student, faculty, staff), managerial, and technical Execution is fast to eliminate excess inventory and technology aging

Principle 14 - Application Become a learning organization through relentless reflection (Hansei) and continuous improvement (Kaizen) Every project has a post-project review. Cyclical upgrades and installations allow for continuous improvement Current management understands that mistakes are learning opportunities

Obstacles to this type of management: Perception: This adds more overhead with no value Moore’s law: The pace of technology change is sometimes not conducive to this type of cyclical planning and approach. “Its not my job”

The Result: Lean management is portable to technology Oakland’s central IT group delivers projects with little or no waste, translating to cost savings understood by other business divisions. The time to deliver has been vastly reduced and internally, the network group has cut installation times by over % Staff members are directly connected to the delivery of services identified as critical to some aspect of the University’s mission or purpose.

Sources The Toyota Way: 14 Management Principles from the World’s Greatest Manufacturer by Jeffrey Liker The Lean Office Pocket Guide. Don Tapping, Publisher. MCS Media