Getting ML.NET to work with SQL Server (SQL Express)

There is a neglected major bug in ML.NET that Microsoft STILL HASN'T fixed.. They must have laid off the wrong team member, and nobody there knows how to fix it, because it is prominent and vexing.
The bug is ML.NET won't connect to SQLServer correctly.  Some have gotten it to work but many can't.
 
After much fidgeting, I was able to get ML.NET to load data from SQLExpress installed on the same PC.  (Note, I still can't get it to work with a normal instance on a different PC, but this is a good start until Microsoft figures out AI can't do everything yet in the programming dept. and rehires the right dude back who can fix the problem.)
 
Here's what I did:
  1. Download and install SQLExpress  2019(it's free)
  2. When installing, select "Mixed Mode Authentication" and "Encryption optional"
  3. Install SQL Management Studio 2022 (this will install as part of the above install package)
     
  4. Open Mgmt Studio and create your new Database, and a new Table, and populate it
     
  5. Open ML Studio: Under Data, click [Change Data Source...]
  6. Select <other...>, and click [Continue]
  7. You now should see ".NET Framework Data Provider for SQL Server" appear in the top "Data source" field
  8. For Server name, you must type /[YOUR SERVER NAME HERE]/SQLExpress
  9. Note: ^ The above does not auto-populate even though it should
  10. Authentication: "Windows Authentication"
  11. Encrypt: "Optional (false)"
  12. Now, the "Select or enter a database name" field will auto-populate Database names.  Pick yours.
  13. Test Connection.  It should be successful.
     
  14. You can now load the table in ML.NET Data 

Getting ML.NET to work with SQL Server (SQL Express)

There is a neglected major bug in ML.NET that Microsoft STILL HASN'T fixed.. They must have laid off the wrong team member, and nobody there knows how to fix it, because it is prominent and vexing.
The bug is ML.NET won't connect to SQLServer correctly.  Some have gotten it to work but many can't.
 
After much fidgeting, I was able to get ML.NET to load data from SQLExpress installed on the same PC.  (Note, I still can't get it to work with a normal instance on a different PC, but this is a good start until Microsoft figures out AI can't do everything yet in the programming dept. and rehires the right dude back who can fix the problem.)
 
Here's what I did:
  1. Download and install SQLExpress  2019(it's free)
  2. When installing, keep instance named "SQLExpress", select "Mixed Mode Authentication" and "Encryption optional"
  3. Install SQL Management Studio 2022 (this will install as part of the above install package)
     
  4. Open Mgmt Studio and create your new Database, and a new Table, and populate it
     
  5. Open ML Studio: Under Data, click [Change Data Source...]
  6. Select <other...>, and click [Continue]
  7. You now should see ".NET Framework Data Provider for SQL Server" appear in the top "Data source" field
  8. For Server name, you must type /[YOUR SERVER NAME HERE]/SQLExpress
  9. Note: ^ The above does not auto-populate even though it should
  10. Authentication: "Windows Authentication"
  11. Encrypt: "Optional (false)"
  12. Now, the "Select or enter a database name" field will auto-populate Database names.  Pick yours.
  13. Test Connection.  It should be successful.
     
     
  14. You can now load the table in ML.NET Data 

What is the Real Job of the Software Development Manager?

There is a longstanding joke in the Agile community that the real purpose of a Software Development Manager is to stay out of everyone’s way.

With self-governing teams interacting with competent Product Owners, an Agile Team can competently produce value in short increments, consistently moving the business further down the road to where it wants to be next. This team collaboration requires focus, discipline, communication, skill, competency, teamwork, and emotional intelligence by everyone involved.

Although humorous, this legitimately does raise the question—While the teams are busy generating value, what is the REAL purpose of the Development Manager?

The software development value stream has often been described as an assembly line of decisions, building a massive mathematical product. The better that humans can harmonize and organize, the better the product’s final utility value and fit will be for the organization’s customers.

Good Software Development is NOT Intuitive

All of this complexity must be managed correctly along the software value steam, and surprisingly it’s not intuitive. What you think you should do next is sometimes NOT what you should do next.  In other words, on-the-job training will produce pretty good managers, but not excellent ones.

The Standish Group’s CHAOS Surveys tell us what a typical organization look like.  Only 31% of projects are delivered successfully. 50% are late. And 19% are complete loses. (Standish 2020). With an combined industry budget of around 250 $Billion, this is a lot of unnecessary waste, dealt out by a lot of mediocre development leaders.  How do we fix this?

Ten Critical Software Development Bottlenecks:

There are ten critical bottlenecks (read: wastes of money) in the software development process and they are not intuitive. I propose the real job of a Software Development Manager is to attend to these things:

  1. Employee Turnover – While this happens in our industry, every time a programmer leaves it takes about 6 months of work for the next one to come up to speed firing on all cylinders. Even highly skilled developers need time to acclimate to the products, the code base, the tools used in that environment, the customers, the customers attitudes, and how they use the product. Figuring a typical developer salary corporate burden today is around $150,000/annually, every time a programmer leaves a company, it costs them about $75,000 and six months of one person’s schedule.  Great Development Managers minimize turnover, saving the company lots of money and unnecessarily lost time.

    A good Development Manager keeps the teams productive and content.  Being near them, praising them for work well-done, and keeping the environment healthy and productive are important factors for healthy teams.  The best environments report high-trust, good communication, happy employees, and a good sense of satisfaction. Some people call this Psychology Safety.  

    It’s important to develop a culture of accomplishment and teamwork.  Team branding with a team identity, vision, mission, goals, and values which are supported by a feeling of appreciation and accomplishment is the magic formula here.

  1. The Hidden Factory: I like to say “If it takes your team eight weeks to get six weeks’ worth of production completed, your team has 2 weeks of hidden factory.” This term describes everything that has to happen over again because it wasn’t done correctly or completely the first time around.  Meeting with customers again for the same reasons, re-structuring screens or reports, changing workflows, re-pushing builds, hot-fixes, testing churn, and many smaller and larger things including multi-million-dollar entire solution rejections which we see occasionally in large companies. These become multi-million-dollar mistakes.

    It’s not easy to address a Hidden Factory problem. Most groups naively think they just need more time to get some things done and if management would relax a bit they could catch up and stop making these consistent mistakes and omissions.  The truth is, they will never address the problem until they start doing work differently.  What they need is a quality system such as CMMI, ISO, or The Stable Framework to solve their reactivity issues.  This is an unintuitive reality of software development.

    Most groups I talk to report about a 33% effort loss to the Hidden Factory over time.  Just do some simple math on the total payroll burden of your Software Department x 33% and that’s only PART of what you are losing unnecessarily.  Add to that the opportunity cost for revenue delays on your products until 33% past the optimum endpoint and the wasted costs magnify.

    It is the Software Development Managers job to bring in a quality system to solve this problem. Quality pioneer W. Edwards Deming was extremely unpopular for putting the blame for low-quality work on management—not employees. Although he was uncomfortable and non-intuitive, he was right.

  1. Not Using AI to Vibe Code: There is a tremendous advantage available now to software development teams who use AI to generate units of code and debug challenging problems. Many groups are reporting about a 30% increase in productivity using these tools. CoPilot, Clive, Grok.com, Gemeni, Claude.ai, and even local LLM’s using Ollama or LM-Studio will provide immediate lift.  To keep up with who’s currently leading the AI wars check out ARCPrize.org.

  2. Using Awkward Tools: The first time I met Ken Schwaber, co-inventor of Scrum, he told me the best Scrum tools were a white board, sticky-notes, and a spreadsheet.  I met him again nine years later on the other side of the planet and he told me the same thing.

    Of course, today, many of us work remotely making those tools awkward. There are many digital tools to chose from. I can tell you from personal conversational experience with many professionals the most popular tools used tool today are Jira (80%), and SmartSheet (20%), with many companies using both.  Probably 10% of the remaining companies use Asana, Shrike, Monday.com, or ADO (Azure DevOps).
    Some companies are even reverting back to a virtual whiteboard and virtual sticky-notes using a group collaboration space such as Miro.com, LucidSpark, or just the new whiteboards inside of Zoom.  This works, and is a fun solution.

    Whatever tools your groups choose, make sure it helps keep you organized for speed, and doesn’t slow down your work. Incorrect tools, or misconfigured tools will slow the team down.

  3. Bad Approaches to Coding: The first part of Robert C. Martin’s book Clean Architecture explains how important correct coding architectural techniques are to minimizing the cost of maintaining a developed system once deployed. Not all coders are the same, and if you hire sloppy coders you will soon have a production product that requires an exponential number of additional programmers just to maintain it.  Data with compelling examples are in his book. Basically, spaghetti code kills productivity.

    This costs a company millions of dollars of unnecessary payroll over time. As an owner, or investor, and because this is non-intuitive, you won’t know what you don’t know until it’s too late—and you will be paying a large team, instead of paying yourself.

  1. Continuing to invest in Monoliths: Software development has evolved. In the past we all invested in Microsoft, Oracle, and other monoliths because they were the mainstream providers of proven technology. “Nobody ever got fired for buying IBM.” or so the expression went.

    While this once was an expensive but safe position, today Microsoft IIS webservers only make up about 4% of the internet.  Open-source solutions like PostgreSQL, Python, JavaScript, are proven, and all you need to construct a complete enterprise, or SAAS solution.  Forgive me for stating the obvious, but the days of these big players are waning.  Don’t spend unnecessarily big money on new investments here. There is no need anymore.

  2. Hiring Wrong: An accountant and a seasoned software development manager would have two very different approaches to hiring for tech talent. While accountants want reasonable value for the best price, Steve Jobs said he witnessed a 25:1 difference in productivity between good and bad developers.

    You want your development staff to match the workload. Staffing for new development should look like an upside-down birthday cake.  You want it top heavy with senior coders, and just a few junior coders on the bottom to do the busy-work. This configuration will yield much more productivity over time of much and more stable products.

    It takes developers years to understand and permanently distill the larger compliment of common logical structures available within most programming languages.  Senior developers look at a challenge and select and organize which “tools” to use to construct the solution.  This is much like making a product from Legos.  Once you have a few years working with them, you’ll command a solid knowledge of all the common shapes.  Developers with this level of competency can be given an objective and can quietly create a solution correctly, the first time.  In contrast, junior developers must search and forage for solutions to what are all-new challenges for them.  This is the process that takes the x25 times longer Steve Jobs was talking about.

  3. Not pulling weeds: Nobody likes a bad apple. Sometimes, despite all initial attempts for reaching across the aisle, you end up with a team member that causes problems for others on the team. This can quickly become a problem that never goes away without intervention. If someone presents with unwanted behavior over time, a good development manager should take the team member aside and find if the problem is temporary. If so, work with the team member privately to help them regain a successful gait. If not, they are likely in the wrong role. Work with them to get them where they would rather by. A good manager wants their team members to be successful not just on the job, but also in life. Good managers understand their team members are also their customers.

  4. Outsourcing Incorrectly: Outsourcing is rarely effective. It’s never really been a good idea. Sure, you can find senior level management who defend outsourcing, but talk to any team and they will tell you it’s awkward, comparatively slow, and frustrating.  Most companies outsource because other companies outsource.  The argument used to be it was cheaper labor, but those days are past. Now the argument is the resources are instantly available.  This is a pretty weak argument. In my experience most senior executives support outsourcing so they can confirm to their senior board of directors, who are even more removed from the real software development value stream, that they are in-fact outsourcing, too.

    Companies who outsource successfully do it a certain way.  They budget for one member of their team to go stay with the outsourcing group for the duration of the project, or at least, go back and forth frequently.  Sometimes the model is reversed and the outsourcing agency has a team member stateside.  That team member spends 9 hours working with the American team, and then the other 9 hours working with the offshore team—each day.  It is brutal, and frankly unfairr

    Studies have shown outsourcing adds 20-40% on to a projects effort in cost and schedule—and that’s if you have a useful product.  Many a local team overcompensates for the outsourced contribution. However popular, it is difficult to outsource efficiently.

  5. Not Investing in Training: Niccolo Machiavelli stated “…training--not budgets--wins wars!” That’s an important principle to digest.  More recently, Deming taught his customers, when evaluating vendors, to ask them how much money they had spent on training their people during the past 12 months.  This will give you some insight into the level of performance you’ll get out of them.

    Deming would draw the effect of training as a tight squiggly line representing effort, which abruptly straightens into a longer mildly jagged line at the point of training.  In this manner he demonstrated how training narrows the amplitude of ineffective effort, transforming the trainees’ efforts into work that move the company down the road towards their objectives faster.

    Training helps both new employees and seasoned employees get better at their jobs.  While new employees benefit from structured guidance on how to perform their tasks, seasoned employees benefit from having real-world experience they can conceptualize into immediate practical application.

                 

    Training also gives trainees a correct vocabulary use throughout the industry, and gives them confidence their efforts and skills are industry-standard.

In summary, statistics show software development is not performed efficently, and these ten factors are keys to leading your teams to perform much better than the Standish CHAOS Report data--or industry average of on 31% of project's being delivered on-time.

An overview of the PMP & CAPM Exams

The PMP and CAPM exams today are 50% Plan-based and 50% Agile.

Find out how to get a free Certified Stable Scrum Master certificate by taking a PMP/CAPM Exam Prep course from Red Rock Research. For details email Contact@RedRockResearch.com or phone us at 801.636.0043.

The exams emphasize 3 sections:

  • Process 50%
  • People 42%
  • Business Environment 8%

PMP: Process 50%

 

The Process portion of the Project Management Professional (PMP) exam emphasizes the technical aspects of managing a project. This domain tests your understanding of project management processes, tools, techniques, and methodologies to ensure project success. It covers the project lifecycle from initiation to closure, with a focus on managing scope, schedule, cost, risk, quality, and other critical project management functions. Here's an overview:

Key Topics:

  1. Executing the Project with Quality
    • Ensuring the project is completed according to plan by executing the work defined in the project management plan.
    • Maintaining quality standards by implementing continuous improvement processes and quality assurance techniques.
    • Managing project changes through effective change control procedures.
  2. Managing Scope
    • Defining and controlling what is included and excluded from the project.
    • Developing a scope management plan, creating a work breakdown structure (WBS), and managing scope changes.
    • Ensuring that all project deliverables meet the defined requirements.
  3. Schedule Management
    • Creating a realistic and achievable project schedule that reflects all project activities.
    • Developing a schedule management plan, using tools such as Gantt charts, network diagrams, and critical path methods (CPM).
    • Monitoring and controlling the schedule to ensure timely completion of the project.
  4. Cost Management
    • Developing a cost management plan that outlines how project costs will be planned, estimated, and controlled.
    • Performing cost estimation, budgeting, and forecasting using techniques such as earned value management (EVM).
    • Managing cost overruns and ensuring that the project stays within its approved budget.
  5. Risk Management
    • Identifying potential risks and developing a risk management plan.
    • Conducting risk analysis (qualitative and quantitative) to prioritize risks based on their potential impact and likelihood.
    • Implementing risk responses (e.g., avoidance, mitigation, transfer) and monitoring risks throughout the project lifecycle.
  6. Managing Project Resources
    • Developing a resource management plan, which includes planning for both human resources (team members) and physical resources (equipment, materials).
    • Acquiring, developing, and managing project teams, including resolving resource conflicts and ensuring effective resource allocation.
  7. Managing Communications
    • Creating a communication management plan to ensure timely and appropriate generation, collection, and dissemination of project information.
    • Managing project communications to keep stakeholders informed of progress and changes.
    • Ensuring that key project information flows effectively within the project team and among external stakeholders.
  8. Procurement Management
    • Planning and managing procurement activities, including contracting with external vendors and suppliers.
    • Creating procurement documentation such as requests for proposals (RFPs) and contracts.
    • Managing relationships with vendors and ensuring that project deliverables from external parties are met.
  9. Stakeholder Management
    • Developing a stakeholder management plan to identify, engage, and manage the expectations of stakeholders.
    • Monitoring stakeholder engagement and making adjustments as needed to keep stakeholders aligned with project objectives.
    • Proactively managing stakeholder issues and concerns.
  10. Integration Management
    • Developing and managing the project management plan, which integrates all project elements (scope, schedule, cost, resources, etc.).
    • Coordinating project activities to ensure that the project is delivered as a cohesive whole.
    • Managing changes through a formal change control process and ensuring the project stays on track.

Core Concepts:

  • Project Phases and Lifecycle: Understanding how the different phases of a project (initiation, planning, execution, monitoring and controlling, and closing) fit together and are managed.
  • Process Groups: The PMP exam focuses on five process groups: Initiating, Planning, Executing, Monitoring and Controlling, and Closing. Understanding how these groups are applied in the project lifecycle is key.
  • Project Artifacts: Knowing key project documents (e.g., project charter, risk register, project management plan, issue log) and their purposes is crucial for managing processes.
  • Agile and Hybrid Approaches: In addition to traditional project management processes, understanding agile and hybrid methodologies and how they fit into the project management landscape is increasingly important.

Exam Preparation Tips:

  • Familiarize yourself with the 49 processes outlined in the PMBOK® Guide and know the inputs, tools and techniques, and outputs for each.
  • Understand the relationship between different processes, especially across knowledge areas like scope, time, cost, and risk management.
  • Practice scenario-based questions that test how to apply specific processes in real-world situations.
  • Understand the integration of agile or adaptive practices with traditional project management processes.

The Process domain ensures that project managers are proficient in managing all the technical and procedural aspects of a project, from initiating it to its successful completion while balancing constraints like scope, time, cost, and quality.

 

PMP: People 42%

 

The People portion of the Project Management Professional (PMP) exam emphasizes managing and leading project teams, engaging stakeholders, and fostering effective communication within a project. This section tests your skills in leadership, conflict resolution, team building, and managing project resources, as well as your ability to inspire and motivate team members. Here’s an overview:

Key Topics:

  1. Building a Team
    • Developing team charters to define team norms and responsibilities.
    • Understanding different team dynamics and selecting team members based on skills and competencies.
    • Implementing appropriate team-building strategies to improve performance and collaboration.
  2. Leading a Team
    • Acting as a servant leader by supporting, guiding, and removing obstacles for the team.
    • Adapting leadership styles based on the team's needs, individual members, and project circumstances.
    • Leading by example and fostering trust among team members.
  3. Supporting Team Performance
    • Monitoring and evaluating team performance to identify improvement opportunities.
    • Providing constructive feedback and addressing underperformance or conflicts within the team.
    • Recognizing and rewarding team contributions to maintain motivation and engagement.
  4. Empowering and Engaging Team Members
    • Delegating tasks effectively and empowering team members to make decisions and take ownership of their work.
    • Promoting a collaborative environment that encourages participation and innovation.
    • Ensuring that team members have the necessary resources, training, and development opportunities.
  5. Managing Conflict
    • Identifying sources of conflict and applying conflict resolution techniques, such as negotiation, mediation, and compromise.
    • Managing interpersonal relationships and addressing issues before they escalate.
    • Using emotional intelligence to navigate difficult conversations and resolve conflicts in a positive way.
  6. Promoting Collaboration
    • Facilitating effective collaboration by encouraging open communication, knowledge sharing, and cooperation among team members.
    • Ensuring that team members understand their roles and the project objectives, creating alignment within the team.
  7. Communicating Effectively
    • Developing and implementing communication strategies that ensure timely and clear communication across stakeholders and team members.
    • Selecting the right communication methods based on the project needs (e.g., in-person meetings, virtual meetings, or reports).
    • Active listening and ensuring that all stakeholders’ perspectives are considered.
  8. Mentoring and Coaching
    • Acting as a mentor to guide team members and provide career development support.
    • Coaching team members to improve their skills and enhance performance in the context of the project.
  9. Managing Stakeholders
    • Engaging with project stakeholders by understanding their needs, expectations, and concerns.
    • Building strong relationships with stakeholders to ensure alignment with project goals and foster support.
    • Managing stakeholder communication, particularly when dealing with competing priorities or conflicting interests.
  10. Creating a Shared Vision
    • Aligning the team and stakeholders around a common project vision and goals.
    • Inspiring team members to work toward the project objectives by communicating the importance and impact of the project.
    • Keeping the team focused and motivated, even when challenges arise.

Core Concepts:

  • Leadership and Motivation: Understanding different leadership styles (e.g., servant leadership, transformational leadership) and how to apply them based on team needs.
  • Emotional Intelligence (EI): Recognizing and managing your emotions and those of your team members to build better relationships and handle conflicts effectively.
  • Cultural Awareness and Diversity: Managing teams with diverse backgrounds, experiences, and perspectives. Creating an inclusive environment that respects differences.
  • Team Development: Using models like Tuckman’s stages of team development (Forming, Storming, Norming, Performing, and Adjourning) to guide team growth.

Exam Preparation Tips:

  • Familiarize yourself with leadership and communication theories.
  • Practice conflict resolution techniques and team-building strategies.
  • Understand the principles of emotional intelligence and how they apply to leading a team.
  • Learn how to manage stakeholder expectations and maintain positive relationships.
  • Be prepared to answer questions about how to handle difficult team dynamics, delegate work, and build trust within the team.

The People domain tests your ability to effectively manage and lead people in a project environment, emphasizing the human side of project management, including communication, collaboration, motivation, and conflict resolution.

 

 

PMP: Business Environment 8%

 

The business environment portion of the Project Management Professional (PMP) exam focuses on the external and internal factors that impact a project, as well as how a project aligns with an organization’s strategic objectives. This section typically covers topics related to organizational strategy, project benefits management, governance, and the influence of various environmental factors on project decisions. Here’s an overview:

Key Topics:

  1. Strategic Alignment
    • Understanding how projects fit into the broader organizational goals and strategies.
    • Ensuring that the project delivers value to the business by supporting strategic objectives.
    • Project portfolio alignment and the selection of projects based on business needs.
  2. Benefits Realization
    • Focuses on defining, managing, and realizing the benefits of the project.
    • Includes creating a business case, benefits management plan, and ensuring that the project delivers the expected outcomes.
  3. Organizational Systems and Structures
    • Influence of the organizational structure on project management, such as functional, matrix, or projectized structures.
    • Understanding how an organization's governance and processes impact project execution and decision-making.
  4. External Environmental Factors
    • Enterprise Environmental Factors (EEFs): These are conditions not under the direct control of the project team but influence, constrain, or direct the project. They can include market conditions, regulatory requirements, and industry standards.
    • How external factors, such as market trends, regulations, legal restrictions, and economic conditions, impact project decisions.
  5. Internal Environmental Factors
    • Organizational processes, policies, and procedures that can affect the project.
    • Consideration of Organizational Process Assets (OPAs), such as templates, lessons learned, and knowledge bases that aid in project management.
  6. Compliance and Governance
    • Ensuring the project adheres to organizational policies, regulations, and legal requirements.
    • Understanding governance frameworks and the role of compliance in project management.
    • Managing stakeholder expectations, including those from regulatory bodies or government agencies.
  7. Risk Management in Business Context
    • Identifying, analyzing, and responding to risks that arise from the business environment.
    • Consideration of external business risks, such as market volatility or political instability.

Core Concepts:

  • Project Business Case: Why the project is being undertaken and how it contributes to the organization’s success.
  • Project Benefits Management: Focused on the realization of benefits and how they are tracked throughout and after the project lifecycle.
  • Stakeholder Impact: Considering the business environment’s impact on key stakeholders, especially external groups like regulators, customers, and suppliers.

Exam Preparation Tips:

  • Understand the strategic purpose of projects and their alignment with organizational goals.
  • Be familiar with how to manage the benefits that a project delivers, including creating a benefits management plan.
  • Recognize how different organizational structures influence project authority, responsibility, and governance.
  • Be aware of the broader business environment and how external factors can affect project success.

The business environment domain is essential for ensuring that project managers see the bigger picture and align their efforts with organizational and external business goals.

 

Move over SRE, we need PRE First

The Toyota Factory in San Antonio has a visitors center where they showcase the Toyota idea of the "Corporate Athlete." They have new employees work simple skill challenges like threading ropes through dowels in a certain pattern within 10 seconds, and give new employees paid time in the gym to work out to build their muscles to match the work.

Also, they pointed out that Toyota's perspective on robots was to use humans with robots, and to minimize the use of isolated robots.

Deming pointed out in the 80's that computers and robots can produce higher quality products, but at a higher cost than advanced quality frameworks like the TPS. Consider a low-end Tesla is $40K brand new vs. a low end Toyota at $20K brand new and that's what Deming was talking about.

The real secret to Toyota quality, and others who have matched them like Ford in the early 2000's and Hyunda in the decade after that is "Prevention over Inspection." This concept people talk about but don't quite understand. It's the same thing as the "Shift Left" movement in IT which is decades late. Deming taught Toyota how to think of every repeatable process step within a value stream as it's own system (System Thinking!) which has inputs, transformational throuhputs, and outputs. Each repeatable step requires it's own quality structure to be risk-managed, so that known past defects can be checked for and detected where they might re-appear. This is the true application of "Prevention over Inspection" and "Shift Left." When understood and applied correctly, the Hidden Factory of waste is detected at the point of entry and eliminated altogether.

There is lots of talk about SRE (Software Reliability Engineering) which also misses this point above about Prevention over Inspection. There are SRE platform apps today that show you problems after the fact--when you hit them, instead of helping you avoid them altogether. The only why to actually achieve SRE is through PRE (Process Reliability Engineering) which precedes SRE and will lead to high levels of SRE if done correctly.

Taken together, we call this SPRE, or Site and Process Reliability Engineering.  The Stable Framework will help you acheive SPRE.

The Stable Framework™: Empowering Information Technology Organizations to Shift Left

In today's fast-paced and competitive digital landscape, information technology organizations are constantly seeking ways to improve software quality, accelerate time-to-market, and enhance customer satisfaction. One powerful tool that enables organizations to achieve these objectives is The Stable Framework™. This new framework supports information technology organizations in adopting a "Shift Left" approach to process quality, empowering them to integrate early, test effectively, and deliver reliable software with greater efficiency.

Understanding Shift Left:

Shift Left is a software development paradigm that emphasizes early involvement of key stakeholders, such as developers, testers, security experts, and operations teams, in the development process. It involves moving tasks that were traditionally performed later in the development lifecycle closer to the beginning. The goal is to detect and address issues as early as possible, reducing the risk of costly and time-consuming fixes later in the process. We call these time-consuming downstream fixes the "Hidden Factory."
Leveraging The Stable Framework™ for Shift Left:

1.Comprehensive Monitoring and Alerting:

Using System Thinking The Stable Framework provides organizations with a robust error detection and remediation system. By continuously monitoring assets and workflow, organizations can identify anomalies or vulnerabilities before they impact the value stream. The framework's process-based toolsets enable the appropriate teams to take immediate action, investigate the root cause, and resolve issues before they escalate.

2.Integrated Testing Environment:

To effectively shift left, organizations require an integrated testing environment that allows for early and continuous testing. The Stable Framework offers guides practitioners to identify repeatable process steps and create quality structures around each. The enables organizations to perform rigorous testing throughout the development process. This ensures that bugs and issues are caught early, reducing the likelihood of critical defects reaching the later stages of development.

3.Incident Management and Root Cause Analysis:

In a shift left approach, incident management and root cause analysis are essential for early issue resolution. The Stable Framework provides incident management features that enable organizations to track, document, and collaborate on incidents from the early stages. By centralizing incident management in a central Process Asset Library within the framework, teams can quickly identify patterns and root causes, allowing them to implement fixes and prevent similar incidents from occurring in the future. This transfers tribal knowledge into institutional knowledge.

4.Collaboration and Knowledge Sharing:

Effective collaboration and knowledge sharing are vital for successful shift left implementation. The Stable Framework facilitates collaboration through its shared Process Asset Libaray, institutional knowledge, and performance console. Teams can collaborate in real-time, share insights, and leverage collective expertise to address challenges early on. The framework also allows organizations to build a centralized knowledge base, capturing best practices, incident learnings, and troubleshooting guides, promoting knowledge sharing and continual improvement.

Conclusion:

The Stable Framework™ serves as a valuable asset for information technology organizations aiming to adopt a shift left approach. By leveraging its comprehensive process definition and improvement system, asset recovery models, incident management capabilities, automation features, and collaboration tools, organizations can effectively shift left and achieve superior software quality, reduced time-to-market, and increased customer satisfaction positioning them for success in today's rapidly evolving digital landscape.

The Stable Framework™: Empowering Organizations with Process Reliability Engineering (PRE) to achieve Site Reliability Engineering (SRE)

In today's digital landscape, where uptime and performance are critical, organizations are increasingly turning to Site Reliability Engineering (SRE) to ensure the reliability and stability of their systems. SRE combines software engineering and operations principles to create scalable and reliable systems and performance workflows. SRE, when done right, requires Process Reliability Engineering (PRE). One essential tool that aids organizations in implementing PRE, and therefore SRE practices effectively is the Stable Framework™. This article explores how The Stable Framework™ provides organizations the ability to "shift-left" and focus on upstream process quality to achieve their SRE goals.

1.Building Resilient Infrastructure:

The Stable Framework serves as a robust foundation for organizations aiming to build resilient infrastructure. It provides a comprehensive set of best practices, tools, and guidelines for designing, deploying, and managing reliable systems, all of which are PRE functions. By following the principles outlined in the framework, organizations can enhance the reliability and stability of their infrastructure.

2.Monitoring and Alerting:

Monitoring and alerting are crucial aspects of SRE, allowing organizations to proactively identify and respond to incidents. The Stable Framework™ offers advanced service and application monitoring capabilities, enabling organizations to gather real-time insights into system health and service performance. By implementing monitoring practices outlined in the framework, organizations can detect issues where they occur and take corrective action before they escalate, or cascade downstream where they become more expensive to fix.  We call this unnecessary downstream flow the "Hidden Factory."

3.Incident Management and Root Cause Analysis:

When incidents occur, efficient incident management and root cause analysis are vital for minimizing downtime and ensuring a swift recovery. The Stable Framework facilitates effective incident management by providing toolset for incident tracking, collaboration, and resolution. It allows teams to streamline their incident response processes, maintain clear communication channels, and track the status of ongoing incidents. Additionally, the framework offers tools for conducting thorough root cause analysis, enabling organizations to identify the underlying issues that lead to incidents and implement preventive measures to avoid similar occurrences in the future.

4.Performance Optimization through Continual Improvement:

As organizations grow, performance stability becomes a critical challenge. The Stable Framework offers process guidance for continual improvemnet and performance optimization techniques. By leveraging these recommendations, organizations can effectively scale their systems to handle increased traffic and ensure optimal performance under varying workloads.

5.Automation and Tooling:

Automation plays a pivotal role in SRE, reducing manual toil and enabling efficient operations. The Stable Framework promotes the use of automation and system-thinking step-based quality management.. Automation practices, such as configuration management, deployment pipelines, and infrastructure provisioning, streamline operations and reduce the risk of human error.

6.Collaboration and Communication:

Successful SRE implementation requires strong collaboration and communication within and between teams. The Stable Framework emphasizes establishing effective communication channels, incident response coordination, and cross-functional collaboration. By adhering to these principles, organizations can foster a culture of shared responsibility and collaboration, ensuring smooth operations and rapid incident resolution.

Conclusion

The Stable Framework™ serves as a valuable resource for organizations embracing Site Reliability Engineering. By following the practices outlined in the framework, organizations can build resilient infrastructure, enhance monitoring and alerting capabilities, effectively manage incidents, optimize scalability and performance, automate operations, and foster collaboration. Implementing the Stable Framework empowers organizations to achieve their SRE goals, delivering reliable, highly available systems that meet the expectations of their users in today's demanding digital landscape.

 

What is the ADKAR Change Management Model

The ADKAR model is a popular change management framework that helps individuals and organizations understand the stages of change, and how to manage change effectively. The ADKAR model was developed by Jeff Hiatt, the founder of Prosci, a leading change management firm.

ADKAR is an acronym that stands for:

  1. Awareness: This stage involves creating awareness about the need for change among the people who will be affected by it. This includes understanding the reasons for the change, the benefits of the change, and the potential impact of the change.

  2. Desire: In this stage, individuals need to have a desire to support the change. This involves understanding why the change is necessary and how it will benefit them and the organization.

  3. Knowledge: Once individuals have a desire to support the change, they need to acquire the knowledge necessary to make the change successfully. This includes training, education, and communication about the change.

  4. Ability: In this stage, individuals must have the skills and ability to make the change happen. This may involve providing additional resources, tools, or support to help people adapt to the change.

  5. Reinforcement: Finally, in this stage, individuals need to be reinforced and rewarded for making the change. This includes recognizing and celebrating successes, and providing ongoing support and encouragement to ensure that the change becomes a part of the organizational culture.

The ADKAR model is a useful framework for managing change because it focuses on the individual level, and helps to ensure that people have the necessary knowledge, skills, and motivation to make the change happen. By following the ADKAR model, organizations can increase their chances of success and achieve their desired outcomes.

PMI's Process Owner vs. Process Manager

PMI (the Project Management Institute) has recently introduced new content into it's curriculum....the Process Owner and Process Manager.  The distinction between these two roles seems to originate from ServiceNow's influence.

In small organizations the same person wears both hats, but in larger organizations these two roles may be split between two people.  Basically the Process Owner is a senior person responsible for "working on" the process, to improve it, while the Process Manager "works in" the process to execute it with efficiency.  A description of each role follows:

Job Description for Process Owner:

Process Owner's are responsible for the end-to-end oversight of a particular business process within an organization. Their main responsibilities include designing, implementing, monitoring, and continuously improving the process to ensure it meets the organization's goals and objectives. Process owners also ensure that the process is compliant with regulatory requirements and industry standards. They work closely with cross-functional teams to identify areas for improvement and implement changes that increase efficiency, reduce costs, and enhance quality. Other key responsibilities of a process owner include:

  • Developing and maintaining process documentation, including standard operating procedures (SOPs), process flowcharts, and process metrics.
  • Monitoring process performance using key performance indicators (KPIs) and other metrics, and identifying areas for improvement. (Note, both roles include this bullet point as it is an ongoing common point of review and discussion.)
  • Leading process improvement initiatives, including process re-engineering and process automation.
  • Collaborating with other process owners to ensure that processes are integrated and aligned across the organization.
  • Communicating process changes to stakeholders, including senior management, process users, and customers.
  • Providing training and support to process users to ensure that they understand and follow the process.

Job Description for Process Manager:

Process managers are responsible for the day-to-day management of a particular business process within an organization. They ensure that the process is executed efficiently and effectively, and that process users comply with the process requirements. Process managers work closely with process owners and cross-functional teams to identify areas for improvement and implement changes that increase efficiency, reduce costs, and enhance quality. Other key responsibilities of a process manager include:

  • Ensuring that the process is executed in compliance with regulatory requirements and industry standards.
  • Monitoring process performance using KPIs and other metrics, and identifying areas for improvement. (Note, both roles include this bullet point as it is an ongoing common point of review and discussion.)
  • Providing training and support to process users to ensure that they understand and follow the process.
  • Identifying and addressing process issues and bottlenecks that impact process performance.
  • Collaborating with other process managers and process owners to ensure that processes are integrated and aligned across the organization.
  • Communicating process changes to stakeholders, including senior management, process users, and customers.
  • Developing and maintaining process documentation, including SOPs, process flowcharts, and process metrics.

Overall, while both roles are involved in managing and improving business processes, the process owner has a more strategic and high-level focus, while the process manager has a more operational and hands-on focus. The process owner is responsible for setting the direction of the process, while the process manager is responsible for executing the process according to the owner's direction. The process owner is more involved in initiating and leading process improvement initiatives, while the process manager is more involved in implementing and monitoring process changes on a day-to-day basis.

The Hidden Factory in IT

The Hidden Factory is everything your group does over again because it didn't go right the first time around.

This ranges from re-doing a failed multi-year project, to re-pushing a production release which had some minor issues the first time around. Sometimes these activities are called "Fire Fighting."

Most groups I talk to tell me that about 35% of their teams efforts are lost to this problem.

Someone must pay for this, and it's very expensive.  Higher prices, lower wages, and lower shareholder dividends are one way to quantify the hidden factory.  In addition, the opportunity cost of not being able to reach your project monetization goals 33% faster means you left money and customers on the table.

The Stable Framework™ is a performance management framework for IT designed to give IT departments the tools needed to tame this wild Hidden Factory beast and bring the fire-fighting down to nearly zero, where it should be.

Read more about it here

Mike Berry