Starting a New Project Efficiently — Technical Side — Makimo Software Development
Establishing a Strong Technical Foundation for Your Project’s Success
As an experienced IT professional, I’ve learned that starting a new software project efficiently requires a strategic and well-thought-out approach. The technical side of the equation plays a crucial role in laying the groundwork for a project’s long-term success. In this comprehensive article, I’ll share practical tips, proven patterns, and insightful perspectives to help you navigate the technical challenges of launching a new project with confidence.
Separating Business Logic from Infrastructure
One of the fundamental principles in software development is to maintain a clear separation between the business logic and the underlying infrastructure or framework. This approach helps to ensure that the core functionality of your application remains focused, maintainable, and independent of the technical implementation details.
In simpler projects, a layered structure with a dedicated business logic layer can be sufficient. This layer should contain the essential problem-solving code, separate from the view or presentation layer. By encapsulating the business logic, you can invoke it from multiple places without the need to repeat the same code.
For more complex projects, the Domain-Driven Design (DDD) approach can be particularly beneficial. DDD emphasizes solving problems within a specific domain, allowing you to prioritize the most critical business requirements while postponing technical implementation details. This strategy helps you maintain a laser-like focus on the core functionality, ensuring that the highest effort is directed towards addressing the most important tasks from the business perspective.
Architectural Patterns: Choosing the Right Approach
The architectural pattern you select for your project can have a significant impact on the development process, the speed of progress, the number of bugs, and in some cases, even the overall success of the project. Carefully and consciously choosing the right pattern, tailored to your specific problem, is a crucial step in setting your project up for success.
The book “Patterns of Enterprise Application Architecture” provides an excellent resource for exploring various architectural patterns and understanding their advantages and disadvantages. Reviewing these patterns can help you make an informed decision on the most suitable approach for your project.
Repositories and the CQRS Pattern
As applications grow in complexity, managing data access across different parts of the application can become challenging. The Repository pattern addresses this issue by abstracting the data layer, providing a clear separation between the business logic and the data access logic.
By encapsulating data access in repositories, you can work with a consistent interface for accessing data, regardless of the underlying data source (e.g., databases, web services, or in-memory collections). This separation also greatly enhances the ability to unit test your business logic by mocking up the data repositories, reducing the dependency on the actual data source.
In some cases, particularly when dealing with large amounts of data or complex read operations, the Command Query Responsibility Segregation (CQRS) pattern can be beneficial. CQRS introduces the concept of independent execution of read and write operations, allowing you to optimize these actions separately and address potential performance issues.
Ensuring Data Integrity with the Unit of Work Pattern
Managing multiple data transactions can become cumbersome and error-prone, especially when dealing with multiple repositories or data sources. The Unit of Work pattern can help solve this problem by keeping track of all data changes during a business transaction and coordinating the write-out of these changes as a single operation.
The Unit of Work pattern works hand-in-hand with the Repository pattern to provide a comprehensive strategy for data access and transaction management. It acts as a buffer that contains all the actions to be performed on the database and executes them as a single transaction, simplifying transaction management and ensuring data consistency and integrity throughout the entire operation.
Handling Failures and Unexpected Scenarios
Providing the best possible end-user experience is crucial, with a focus on fast reaction times (users are most satisfied when any feedback is returned within 200 ms) and the absence of unexpected crashes. During the development of a Minimum Viable Product (MVP), there’s a high chance that something has been missed, not foreseen, or wrongly assumed, such as an external service stopping to work.
To address these challenges, the Circuit Breaker pattern can be a valuable tool. By using it, you can ensure that potential failures, even in independent parts of your system, will not cause catastrophic effects, and the user will be promptly informed about the problem.
Avoiding Microservices at the Start
I advise against introducing microservices at the start of a project without a clear reason. Such an approach brings communication solutions that are not necessarily easy to implement, may turn out to be slower than regular function invokes, and may be poorly partitioned. Complexity and communication issues rarely pay off outside of mature projects. At the initial stage, it’s not always clear what to portion out as an individual service, so it’s better to wait with this decision.
Leveraging Design Patterns and SOLID Principles
It’s important not to lose control over attributes, the parameters passed to them, or their dependencies. Otherwise, the code situation will get worse and worse, leading to unnecessary complexity and unclear code relations. A popular option to avoid this is to apply SOLID principles, which provide a set of well-established guidelines for writing clean, maintainable, and testable code.
Additionally, design patterns can be a valuable resource. These are well-tested and proven solutions to recurring problems in software construction. A collection of such design patterns can be found in the legendary book “Design Patterns” by the Gang of Four or on the Refactoring Guru website.
Embracing Automated Testing
Quick and reliable automated testing of new functionalities is not an option in current professional standards — it’s a must-have. Although it costs developer time, it’s a huge productivity boost for developing new features later. The main reason for this is the confidence that a bug has not been introduced to the existing functionality, which brings satisfaction and stability to the end-user’s experience.
When writing tests, it’s important to focus on testing code behavior, not internals, so that your tests will not need to change each time you’re refactoring a couple of things inside your module.
Maintaining Code Aesthetics and Formatting
It’s important to ensure the aesthetic formatting of the code so that everyone on the team can read it comfortably (after all, code is read more often than written), and also that the code itself appears coherent and aesthetically pleasing. Regardless of the programming language used, there are automatic formatters available that are easy to configure and even simpler to use.
Comprehensive Documentation: The C4 Model
Documenting the architecture and code of a software project is crucial for ensuring that all team members can understand and contribute to the project effectively, as well as help to hand over the project. One highly recommended approach for achieving clear and comprehensive documentation is the use of the C4 model.
The C4 model focuses on the software architecture, providing a framework for describing the system at different levels of abstraction. This includes the Context level (system relationships with users and other systems), Containers (the high-level technology choices), Components (how containers are divided into components), and Code (details of the components’ implementation). By documenting a project through these four levels, teams can achieve a better understanding of the system’s architecture, making it easier to maintain and extend.
In addition to the architectural documentation, it’s crucial to include detailed information about the backend entry points. Utilizing tools such as Swagger (now known as OpenAPI) for this purpose can significantly enhance the documentation process, providing interactive and user-friendly interfaces for understanding and accessing the API.
Streamlining the Development Environment
Easy-to-follow procedures for setting up a development environment, quickly finding development credentials, or knowing your issues all come in handy when working within a team. A proper place to insert such information is the README.md file, which provides an overview of a software project, including instructions, information on how to install, use, and contribute to the project.
Capturing Architectural Decisions
Architectural Decision Log (ADL) most commonly consists of short Markdown files called ADRs, which contain a particular decision made across the project, such as why a specific database was chosen and not another one. These documents help to memorize and record the rationale behind specific decisions, becoming valuable over time.
Embracing Containerization and CI/CD Practices
Developing software often requires the use of multiple languages, frameworks, and discontinuous interfaces between tools. Configuring and sharing these elements can be greatly simplified with the help of Docker, which streamlines and accelerates the development workflow. Moreover, most solid cloud platforms provide support for Docker, allowing you to use it to configure and run production environments as well.
Incorporating a well-defined code pipeline that runs whenever changes are introduced to the codebase or other criteria are met is crucial for automating and streamlining the processes of building, testing, and deploying applications. Continuous Integration/Continuous Deployment (CI/CD) practices, often realized through pipelines, provide a structured and automated pathway for code changes to be systematically prepared for production environments.
A Template for Efficient Project Kickoff
To provide a practical example of the principles and techniques discussed, I’ve created a Python template repository that follows most of the practices and rules described in this article. The repository presents a simple example of DDD modeling, where effort has been put into solving a specific business task. The business logic is independent of the framework, allowing for easy modification and further development with any redundant dependencies.
The repository demonstrates the separation of commands and queries (CQRS) to address potential performance issues or complex queried data. Database sessions are controlled via a manager, automatically handling potential connection issues. The entire project is dockerized, enabling easy launch on any computer with Docker installed. Automated tests are included at the unit, integration, and end-to-end levels, ensuring the reliability and robustness of the codebase.
This template is available under the MIT license, allowing you to use it without any financial obligation. You can find the link to the repository at the end of this article.
Balancing Principles and Practicality
Starting a new project efficiently from a technical perspective requires considerable and well-thought-out decisions to avoid redundant frustration, workarounds, and hacks. Unfortunately, there is no universal guide with a simple path for every software system. None of the patterns and techniques described in this article are silver bullets and should not be followed blindly.
It’s worth considering applying the principles and patterns described from the very beginning, to the extent in which they provide tangible benefits. With the right balance, it will pay dividends both to the development teams and stakeholders. In some situations, the quality of the code and its architecture don’t matter, for example, when the program will be used only once. However, most systems require changes and continuous development of new functionalities, and clear code with well-chosen solutions definitely facilitate this process.
By following the guidance and insights provided in this article, you’ll be well on your way to establishing a strong technical foundation for your new software project, setting the stage for long-term success and sustainable development. Remember, there is no one-size-fits-all solution, but the principles and practices outlined here can serve as a solid starting point for your journey.
For more information and the link to the Python template repository, please visit https://itfix.org.uk/.
