As organisations start moving, however fast or slow, towards a DEV-OPS model with owning the full stack together with Agile methodology, it would be the perfect time to mould the new role of the architect for this new world. Unfortunately, though, it’s not that easy.
It makes us beg the question of what is an architect nowadays?
An architect is an engineer with a holistic view, so they are required to look well into the future, while also paying close attention to what is happening right now.
Too fuzzy? Let me make it clearer.
The architect thinks strategically and tactically and their role is to maintain the skeleton and consistency of an application, keep the application efficient and at the same time flexible. It is a challenging role because the application or the whole enterprise sometimes swings between optimization and flexibility (read as innovation) cycles.
Even writing about it isn’t easy, as I have spent quite a bit of time thinking about writing this and then re-thinking the words over and over again.
In my head, I keep wondering: will it cause too much noise? is it too controversial?
But it’s a conversation I believe we need to have as a community and I’m curious to see the outcome. And guess what...in order to stimulate the discussion, this will be the next topic of my talk at the ECS conference in London on 24th September.
I've recently been involved in a number of projects that are moving at a relatively fast pace, and are sprinkled with Agile and DevOps in some parts of the organisation.
Having covered both architectural and engineering roles, I found this opportunity both challenging and intriguing.
I will use the words DEV-OPS and Agile loosely and almost interchangeably. Those of you who are familiar with the subject might call this a crime, but this post does not focus solely on the role of the architect in this weird world.
Lately, I have been asking myself a lot does an architect fit in an engineering-focused agile organisation?
To find the answer, I've been trial running a series of different approaches, and hopefully, this article will help me work it out as well.
Another question that immediately comes to my mind is how does an organisation balance traditional governance with the journey, and the intricacies of DEV-OPS and Agile?
Needless to say, transformation is often a transitional phase of chaos and uncertainty. This is when a team should move from a waterfall model of governance to a traditional waterfall, and then to a more DEV-OPS approach.
In order to achieve this, the architects apply governance and measure the amount of architecture governance applied to DEV-OPS.
The Ideal World
The ideal world of cybersecurity would have a core of hybrid engineers and architects to focus on the various different security topics.
The engineers, and more specifically the application owners, are the herald of the application, defenders of the structure and the budget allocated to the application, or service.
The business would then empower the DEV team to develop the applications securely.
Governance is linked to the concepts of trust and auditability. Nonetheless, in order to provide assurance in the most efficient way, it should employ automation as much as possible.
The automation part should be built into the security pipeline through elements like: :
Automated code scanning, although this will need fine-tuning
Automated security testing and hardening of the base images
Not promoting code into production if it contains a number of critical or high vulnerability
Automated 3rd party library check
It’s worth noting that automation is not always fun and games and the report that comes from the automated tools still has to be triaged.
On top of that, the code would have some principles, standards and best practice (coding) confirmed at the beginning of the design phase.
This would be the ideal case scenario, where some of the elements can actually be developed internally while some others can be purchased externally, and then they can be integrated logically and organically.
In this ideal world, the architect and the application owner would team up and review the lifecycle of an application.
The role of an application owner is to defend the budget and ultimately decide on the risk assessment.
The role of an architect in this ideal world is to oversee the number of changes, forecast structural changes, and ensure that the application foundations are not only solid but also that they are not endangered by too many changes.
Like in any real-life architecture building or application that undergoes drastic renovations and modifications, this might end up threatening the logic of the application, thus questioning its principles? The quick-change enables fast delivery, but it is not always optimal or efficient. The role of the architect therefore is to ensure that the application remains efficient.
So in this fast, ever-changing world, how does the security architect ensure there is governance?
That’s simple. The security architect should team up with the application owner and the application architect. The security architect would then work on structural changes that may or may not impact the security of an application (e.g. logic of access control).
The other focal task of the security architect is to review the roadmap of possible application changes and assess them from a security perspective.
The architect would also work in conjunction with the application owner/solution architect to assess the proposed vulnerability mitigation.
Nonetheless, this shall be an exercise to be done in conjunction with the education of both the application owner and the solution architect.
One of my mantras is “security is everybody’s job” and one of the main reasons I say that is because the security architect would not be able to scale up if they do not delegate, at least a bit.
Agile Methodology
I have always been intrigued by the Agile methodology and have read a fair bit about the subject.
A really good reference book is "The Art of Doing Twice the Work in Half the Time" , aka the Scrum methodology by Jeff Sutherland and his son JJ.
Without giving too much away the Agile/Scrum methodology (again, I will use the two terms interchangeably), is in essence about effective prototyping.
I can already hear a lot of criticism coming my way for oversimplifying it. I know, I'm sorry, but I am actually trying to oversimplify the subject.
The theory consists of developing something and constantly refining it, and this goes completely against the old waterfall method, which demands that all requirements are defined at the very beginning.
Actually, to be precise, let me quote the definition of the Agile Method.
Definition
An Agile Method is a particular approach to project management that is utilised in software development. This method assists teams in responding to the unpredictability of constructing software. It uses incremental, iterative work sequences that are commonly known as sprints.
Based on their combined experience of developing software and helping others do that, the seventeen signatories to the manifesto proclaimed that they value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is to say, the items on the left are valued more than the items on the right, which is a very different approach to that which was previously used.
My take on the security challenges of the Agile Method
I'm quite fond of the Agile Method, because, in my experience, it works the best and promotes small incremental changes throughout the development of the project, rather than a big session at the beginning where every possible requirement has to be defined.
Having said that, as an architect, I find this methodology sometimes challenging in my role, as it tends to sometimes oversimplify critical elements of the architecture, thus making them all flat for the sake of fast delivery.
Actually, after reading the previous sentence again it sounded a bit fuzzy to me, so let me try to clarify it a little bit more.
I have found throughout my career that both critical decisions and designs of critical pillars of an infrastructure or a project requires a lot of thinking and definition of the key requirements. The Agile development tends to kill this thinking, but this is just my opinion, so if you disagree then please tell me.
I also found that the perfect solution might sit somewhere in the middle of the two methods - agile and waterfall.
An architect always need some oversight on where an application should be (and become) instead of just a few weeks of change windows.
The best hybrid that I’ve seen is when architects constructed an application foundation and then the engineering team worked on sprints with user stories. In this hybrid, the solution/application architect would oversee the construction of the application and ensure it followed the path defined in the strategy.
When big projects or transformations are initiated I have found that time is not always dedicated to defining the foundation architecture and the strategy, this is not done in agile as it requires actual time to think.
Those key decision elements need to be well thought and refined as the project goes along (possibly with the agile philosophy of prototyping). I have found this method helps to speed up the whole project in the long run while still preserving the key function, structure and foundations of the architecture.
Initially, the method might result in slower and more cumbersome progress (depending on the size of the project) but it helps, in the long run, to build and create a solid foundation for the project.
In the second part of this article, I will be looking in more detail at the security challenges posed by the Agile Method and how you can decide what is the best option for your business.
In the meantime, if you’d like to discuss any of the issues I’ve mentioned in this article let’s chat or get in touch on Twitter @FrankSEC42, or on Linkedin Francesco Cipollone.