Introduction
Gone are the days when organizational leaders solely shaped social structures while technical leaders focused on technical aspects. Welcome to the present, where we engage in projects that intertwine social and technical elements. Modern projects are characterized by complex socio-technical systems, where no significant problem can be attributed solely to technical or social factors. In fact, all problems and solutions involve both social and technical aspects. This reality challenges us to adopt a holistic approach to understanding and transforming socio-technical systems. This is the essence of socio-technical engineering.
In this series of articles, I will guide you through the intricacies of socio-technical systems, such as software projects. I will shed light on the structure of these systems and highlight the interrelationships between social and technical elements. By doing so, you will gain a better understanding of how to create, comprehend, and transform socio-technical systems effectively.
In this first article, let's explore what makes socio-technical systems so complex: the four primary sources of complexity.
Source Number One: Business Domain Complexity
Business domains do not come with pre-defined entities and explicit use cases. Extracting and distilling knowledge about a business domain is a challenging process that involves uncertainties, ambiguities, and unknowns. Decisions must be made based on incomplete and sometimes inaccurate information. Questions arise, such as how to carve out working packages, how to adopt an agile design approach when the end product is uncertain, or how to hire personnel with a vague scope in mind. The tension between fixed constraints like time and budget and the fluid nature of product visions contributes to a significant level of complexity for decision-makers.
Example
Imagine a project as a pipeline of artifacts: customer value proposition -> solution design -> system requirements -> system design -> technical specification -> working packages -> code artifacts -> QA artifacts -> CI/CD artifacts -> product. Each step in the pipeline relies on inputs from the previous steps. The complexity of the business domain introduces uncertainty right at the beginning of the pipeline, in the customer value proposition and solution design. This uncertainty propagates downstream, forcing all stakeholders to make decisions amid uncertainty and prepare structures that allow for flexible adjustments when inputs are clarified or changed.
Source Number Two: Technical Complexity
Individually, each technology may be complicated, but not necessarily complex. However, when multiple technologies are combined, each with its own demands and limitations, the complexity of the software stack increases. Engineers must carefully consider how their decisions impact the utilization of all technologies working together. Furthermore, different stakeholders often have their preferences when it comes to technical decisions, which further complicates matters. The ongoing tradeoff between the local complexity of system components and the global complexity of the overall system fuels debates such as microservices versus monoliths.
Example
Technologies with high levels of abstraction, like Node.js, relieve engineers from extensive manual work. However, they come with infrastructure resource demands and performance trade-offs. These demands may necessitate an architect to segment the system into smaller components, thereby increasing the system's overall complexity.
Source Number Three: Operational Complexity
Operating a car is relatively simple because you usually have a clear understanding of your location and destination. However, operating a software project involves fragmented information about the current situation and an uncertain understanding of the future. Tasks are interdependent, and uncertainty and opacity in one task can impact the operational complexity of its dependencies. Unforeseen events, both social and technical, can occur. People may unexpectedly leave the project, budgets might be cut, and neglecting technical debt can lead to significant troubleshooting efforts.
Example
In some cases, projects become overwhelmed by bugs. This situation often presents a dilemma: do we quickly fix the bug to unblock the upcoming release, or do we take the time to redesign the system and eliminate the root cause of the bug? This choice poses a challenge, as opting for short-term benefits simplifies the current situation but defers problems to the future, thereby increasing operational complexity.
Source Number Four: Political Complexity
Political complexity is frequently overlooked, but it significantly contributes to the overall complexity of a project. People are political beings with their own goals and interests, including the pursuit of influence, security, and personal comfort. Individual attempts to improve one's position may clash with similar attempts by others. The strategy of making the pie larger is rarely applicable. Usually, the pie is as large as it can be, and everyone strives for a bigger piece. Frictions and conflicts are inherent in groups and add complexity to any project. Ignoring or avoiding conflicts only amplifies political complexity. While peaceful behavior is desirable for a project's culture, indecisiveness can be detrimental to managing complexity. If people are uncertain about a leader's ability to make tough choices, they may lose motivation to actively contribute.
Example
It is in the interest of service providers to associate themselves with successes while distancing themselves from failures. Consequently, service providers often present information in a way that favors their reputation. By doing so, they can protect themselves from contractual underperformance. However, such behavior hampers transparency and contributes to complexity. We commonly refer to this behavior as politics.
Conclusion
Modern technical workplaces have evolved into socio-technical systems, where social and technical decisions and structures are interwoven. These systems are highly complex, primarily due to four major sources of complexity. Each source arises from tradeoffs, constraints, dependencies, and conflicts: the tension between a vague scope and fixed budget, conflicting interests of individuals, limitations of different technologies used together, and the challenge of balancing short-term and long-term benefits while navigating opaque dependencies. Understanding these conditions is fundamental to socio-technical engineering. To gain this understanding, we must approach and treat socio-technical systems as actual systems with requirements, designs, and bugs. In the next post of this series, we will delve deeper into these aspects. Stay tuned...
Comments