Just got back from SD West and unpacked our Jolt Productivity Award. Huge credit and thanks to all the users that provided the stream of feature suggestions that has contributed to making Structure101 such a great product. And congratulations to the product and development team at Headway who spent so many tortuous hours discussing and honing each new feature (sometimes politely!) so that the product stayed usable as it got deeper.
... in the Design and Modeling category. Yeah!
Just released, Version 3.1 adds lots of new stuff to the Architecture perspective to make it much easier to discover the current structure and move classes or packages around to define a preferred architecture.
First thing is a simple expand and collapse button on each cell. So for example you can ask Structure101 to create a high-level architecture diagram from the current code-base - no need to worry about how deep to make the initial diagram since you can now expand and collapse cells once you have the initial diagram.
Let's say you get the following diagram:
This shows a layering violation from the strut2 package to the dispatcher package. Click the "+" to find out what is being used at the next level of detail.
So something in struts2 is using something in dispatcher.controller. Expand both cells...
And we see there is a single class-to-class dependency causing the violation. Also, since the Dispatcher class is below the other classes in the dispatcher.controller layering, we know that the other classes use, but it does not use the other classes. And the layering in the struts2 package indicates that we can move Dispatcher to the indicated level to fix the violation without creating new ones. Just drag and drop Dispatcher to create this:
Collapsing the 2 top-level cells by clicking the "-" icons shows the original layering but with the underlying code rearranged so there is no longer an architecture violation.
Other stuff that has been added in this version:
Final note, if you are using the architecture diagrams to drive your developers, you should choose the expansion level of cells consciously before publishing them to the project repository. Their code changes will be checked against the visible levels only.
I have updated the architecture diagrams for the just-released Spring 2.5. Any new or changed packages are highlighted (since 2.0.6). The diagrams are also online - if you pointed your IDE plugin at these after my previous entry, you will be seeing the updated diagrams in your IDE already, and any compile time messages about architecture violations will be based on the new versions.
Here's the new top-level architecture:
And here are the internals of the larger subsystems:
org.springframework.aop:
org.springframework.beans:
org.springframework.jdbc:
org.springframework.jms:
org.springframework.orm:
org.springframework.web:
Released today, the new version 3 capabilities make Structure101 a nicely rounded architectural control solution in addition to the previous structural analysis and complexity measurement capabilities.
For example:
Also, there's a new online demo (about 13 minutes, with audio (me!)) and the version 3 Help is available online.
Enjoy!
A lot of work went into this. A "periodic table" of visualization methods for data, information, concepts, strategy, metaphors, process and structure.
Here's a screen shot - be sure and visit the original if you're interested - when you mouse over each cell, you get an example of the corresponding visualization method.
I didn't see any of the visualization techniques used by structure101 for visualizing software dependencies and architectural layers. It is more focused on business processes, though Data Flow (Df), Entity-relationship (E) and Flowchart (Fl) diagrams are there.
Java packages are often used like file-system folders to organize source. But source files differ from "normal" files in that they are highly inter-dependent. Considering this interdependence as a package hierarchy evolves can have significant productivity benefits.
Packages as Folders
Java packages provide an ideal way to organize code into a scalable, hierarchical structure that helps us find specific code.
In this sense, packages can be used like folders in a file-system:
Often Packages are only used as a kind of filesystem equivallent. However the package hierarchy can also be used to reinforce the intended design and associated development activities.
Packages as Design Abstractions
Source files differ from other files typically stored in the file-system:
The aim of a package design should be to support these characteristics. For example, the design could explicitly support Martin's “Reuse/Release Equivalence Principle (REP)” (article, book) whereby packages are developed, built, tested and released against released versions of the packages upon which they depend.
Design is not something that happens once at the start of the project – it is an activity that spans the life of an application or product. This fact has become explicit with the iterative and Agile development models. As the code-level design continues, the package-level design emerges.
Unfortunately, the emergent design is often invisible and so forgotten. Not only does the original design degrade, but the overall structure tends to become excessively complex. As the supporting structure dissolves, development activities become harder and the cost of each new feature increases.
This priciple of emergent design is important here. Clearly when I have a project of 50 classes in a half-dozen packages, the overhead of a sub-optimal package dependencies isn't going to slaughter me. But if my project is going to grow to 5000 classes, then putting in minimal effort from the start can save huge effort when things get more complicated.
In part 2 I'll take a closer look at cyclic package dependencies and why they matter.
Structure101 lets you work with both structure (the whole code-base as it is) and architecture (the subset of the structure that you really care about, and how it should be). It lets you define the architecture in the context of the physical structure and diseminate this to the team. Architecture diagrams are what makes this possible.
Layering and composition
Structure101 architecture diagrams use a concise visual notation for representing architectural layering and composition. Here is an example of one of the architecture diagrams that we use for the structure101 code-base.
The principle is simple; components (“cells”) should only depend on components at lower levels, not in the same or higher levels.
Layering Overrides
Sometimes a top-down dependency structure is too simple to capture the intent of an architecture. "Overrides” allow you to override the default layering of a diagram. For example we may decide to allow a specific dependency from a cell to a higher-level cell. The override is shown as a green (“allowed”) arrow on the architecture diagram. (Note that enabling this “upward” dependencies practically merges the “hiView”, “xbase” and “graph” components from the perspective of testing, reuse, development, etc.)
A more common example is where we wish to enforce a more strict layering. For example we may want one layer to only use the next layer down, but not layers below that. Such an override is shown as red (“disallowed”) on the architecture diagram.
Combining diagrams
It is not necessary to include all aspects of an architecture on a single structure101 architecture diagram.
A common scenario is where a number of “add-ins” are distributed across several packages. For example, this diagram shows part of the structure101 architecture.
It is correct, but incomplete. Classes in assemblies.X should never depend on classes in lang.Y. We could express this by adding several overrides, but it is much cleaner to use a separate diagram for this aspect of the architecture.
The next diagram defines a number of “language packs” that do not have a direct equivalent in the physical structure (they are “pure” architecture components), but express the architectural constraint that was missing above.
The combination of the 2 diagrams defines the intended architecture.
Mapping the architecture to physical code
In order to understand how a physical code-base conforms to an intended architecture, we need to map the architectural components (cells) to physical code.
Simple patterns are used to establish this mapping. This has a number of benefits:
Another flexibility is that a physical entity maps to the component with the most specific pattern. For example if I include 2 components, one with com.headway.lang.* and the other with the expression com.headway.lang.java.*, then the class com.headway.lang.java.myClass will map to the latter. The effects of this can be at the same time subtle and powerful. For example I could move the component that maps to com.headway.lang.java.* into another “parent” altogether.
Finally, each diagram has a (possibly empty) expression that maps to “excluded” items. This is useful if some physical entities would otherwise undesirably map to a component in the diagram.
Once the mapping is established, any dependencies that violate the architecture is shown on the diagram as a curved dotted line as shown here between component “graph” and the higher-level package “hiView”.
It is easy to discover the code-level cause of a violation by selecting it on the diagram within a structure101 client or IDE plug-in.
This update now checks for architecture violations automatically when you do a build (previous version was "on-demand" only). More on structure101 IDE plug-ins.
In response to O’Reilly's just-published Beautiful Code, Johnathan Edwards explains why he couldn't go along with the premise. One sentence in his excellent piece stood out for me:
"The human mind can not grasp the complexity of a moderately sized program, much less the monster systems we build today."
This is true, but only to a point. Clearly it is humanly impossible to understand the whole "design" of a million-line code-base by studying just the lines of code. But hopefully that is not necessary.
If it's written in an OO language, then I'm mentally constructing class diagrams as I read the code. Much better if these are visible to me - I can work on understanding the class-level structures, dipping down to the code-level as I need to.
I'm still not going to make sense of thousands of classes as a single conceptual group, But that's not what I'm going to do. I'll start organizing the classes into groups. Ideally this was done as the code-base evolved and I have some physical representation of these groups to help me with my task. In Java, the package structure largely serves this purpose. Understanding the package hierarchy and interdependecies (in conjuction with dipping down to the class and code-levels) is not going to be a cake walk, but if the hierarchies are well-structured, it is surely possible.
The degree to which my million line code-base can be understood is therefore largely dependent on:
1 is pretty easy to measure (a lack of cyclic dependencies; not too much complexity at any point of breakout in the hierarchy). But being measurable, I'd call this "quality" rather than "beauty".
2 depends on 1, but goes a step further. It requires the inspired human touch. Herein lies the beauty.
The bottom line is that structure and architecture are an intrinsic part of the code, and any discussion of code "beauty" without them isn't going to work for today's monster systems.
