Design Reading Techniques 2018
In a software development process, the development of software architecture and design is between requirements engineering and code implementation. Thus software design needs to satisfy a requirements specification and provide guidance to subsequent implementation. As far as software design inspection is considered, a reasonable assumption is that the requirements specification as input is free of error. Of course, any issues with the requirements specification uncovered during design inspection shall be captured and dealt with.
The focus of design reading and inspection is, however, to make sure the design is correct and complete with respect to requirements and the design itself is consistent and clear and has adequate information to start the subsequent detailed design and implementation. Design reading or inspection is crucial since the design quality directly affects the quality of and effort required for the implementation.
In addition to generic reading techniques such as ad hoc, checklist-based, and perspective-based reading discussed in earlier blogs, this blog introduced specific reading techniques for design inspection. Usage-based reading employs prioritized use cases to focus readers’ attention on what matters most to end users.
Traceability-based reading is applicable to inspecting OO designs. The family includes seven techniques organized as horizontal reading and vertical reading, with the semantic checking as a theme across both.
Horizontal reading ensures all design artifacts are consistent among themselves and vertical reading ensures designs are consistent with the requirements specification. Architecture reading is an emerging reading technique, motivated by the ideas behind Traceability-based reading. While these reading techniques focus on defect detection in software artifacts such as design and architecture, scope-based reading is meant for developers to read the application framework and figure out how to construct software applications.
With these ranges of reading techniques, you are ready to read and understand various software design artifacts to accomplish various tasks, be it for defect detection or for constructions.
One of the reasons to read software artifacts is to detect defects early in a development process. However, not all defects have the same impact on end users.
Thus the goal of software reading should not be to find as many defects as possible but to find the most critical defects that matter to users. Usage-based reading is developed just for that. A similar idea is employed in the operational profile-based software testing, which greatly improves software reliability.
Usage-Based Reading Technique
The main idea of usage-based reading is to use prioritized use cases to drive and focus an inspection effort on detecting critical defects that matter most to end users. Use cases are generally developed as specifications of a system. A use case is a set of scenarios together satisfying a common user goal, where a scenario is a sequence of steps describing an interaction between an agent (user) and the software system. Usage-based reading focuses on software quality from a user’s perspective.
The usage-based reading is schematically, where the artifact to be read and the list of prioritized use cases are input, and the list of defects is the outcome of the reading. Depending on how many use cases a system supports, prioritization can be done in an ad hoc way or systematically through, e.g., a pairwise comparison using the analytic hierarchy process. It is important to note that prioritization shall be done by prospective users or someone who is familiar with the usage of the system.
Two schemes, time-boxed and rank-based, can be employed in usage-based reading. In the time-boxed reading, one allocates a fixed amount of time to the whole reading and divides the allocated time to individual use cases proportionally by their priority weights. Thus an important use case will be allotted more time for a close examination.
Alternatively, a rank-based reading can be performed. That is, use cases are examined in their priority order and there is no forced time limit except the whole allocated time. The basic steps of a rank-based reading are listed in Panel 1: Instructions for Usage-Based Reading (Rank-Based).
PANEL 1: INSTRUCTIONS FOR USAGE-BASED READING (RANK-BASED)
1. Prioritize and sort use cases in order of importance from the end users’ point of view.
2. Select a use case with the highest priority.
3. Trace and manually execute the use case scenarios through the document under inspection.
4. Ensure the document fulfills the use case goals (the needed functionality is provided, the interfaces are correct, etc.). Identify and report issues found.
5. Select the next use case with the highest priority among the remaining ones and repeat steps 3 and 4, until the allocated time is used up or use cases have been exhausted.
Use-case is the common instrument that helps to focus on users’ needs. In usage-based reading, prioritized use cases are used to focus the defect detection effort. Use cases are used or involved in other reading techniques as well. For example:
Perspective-based reading can have a user’s perspective, where a reader, who takes the end user’s perspective, creates a user’s manual or use cases during reading.
A reader performs a vertical reading, which is a technique in the Traceability-based reading family and compares design diagrams with use cases to ensure the design is correct and complete. Traceability-based reading is discussed in the next section.
The predominant application of usage-based reading is design reading, although it has been applied to requirements documents as well (see the empirical experiences below). It is our opinion that it is a bit awkward to apply usage-based reading to requirements, however. A textual description of requirements is typically developed first, followed by a use-case document.
Upstream documents are frequently used to guide the development and review of downstream documents. When used as guidance, upstream documents are assumed to be correct. So employing use cases to guide requirements inspection has a limited value in practice.
Prioritized use cases can be used to direct code reading, e.g., which areas readers focus on and put more effort in. Once a piece of code is selected, a reader tends to have a local view of the code and its functionality implemented, not a global view such as the priority order of use cases.
The sheer amount of code in a project is typically much more than an individual can deal with in a reasonable time. It is thus unproductive to apply usage-based reading to code documents. We, therefore, discuss usage-based reading as a specialized reading technique for design documents.
Variations of Usage-Based Reading
In the usage-based reading developed by Thelin, use case prioritization is performed by an expert of the end system. Winkler, however, suggested that use case prioritization can also be done by readers individually when reading software artifacts, based on one’s own understanding of the application domain.
Cantone reported a use-case-driven reading technique for analysis and design of Unified Modeling Language (UML) diagrams. This is not the same reading technique as that for OO code reading, although they have the same name; rather, it is closely related to usage-based reading with the exception that uses cases are not prioritized. Analysis and design artifacts under review include a vision document, a use case diagram, a view of participating classes, and sequence diagrams.
Four checklists were developed, one for each artifact type, that specify what issues to look for. In addition to checklists, the other idea behind the use-case-driven reading is to specify how to find defects in artifacts ranging from the informal specifications down to use-case realizations and sequence diagrams, using use cases as a guiding light.
Their procedure guides a reader to examine the entire set of artifacts, following the order in which they are created using a development process, such as the Rational Unified Process. The authors compared the reading technique with a checklist-based one and concluded that checklist-based reading was more effective.
In terms of defect detection rate, checklist-based reading peaked later but at a higher value. The authors indicated that the difference was not statistically significant, however.
Usage-based reading has been tested on design documents in academic settings where subjects were software engineering students. The effect of use case prioritization in the usage-based reading was assessed first. Defects were classified as critical, major, and minor.
Readers found different defects using prioritized use cases compared to randomly ordered use cases. Readers who used prioritized use cases detected more critical and major defects than readers who used randomly ordered use cases, and the difference was significant.
When all defect categories (i.e., critical, major, and minor) were considered, the difference was not significant anymore, which suggested that prioritized use cases enable readers to detect important defects from the users’ point of view. Readers using prioritized use cases were also more efficient, i.e., they detected more defects per hour in areas of all defects, critical defects, and critical and major defects.
Usage-based reading (with prioritized use cases) was compared to checklist-based reading. Researchers concluded that usage-based reading was more effective and efficient than a checklist-based one. Readers using usage-based reading detected 75% more critical defects than readers using checklist-based reading, 51% more critical and major defects, but were not significantly more effective in detecting all defects.
Readers using usage-based reading detected 35% more total defects per hour than readers using checklist-based reading, 95% more critical defects, and 70% more critical and major defects. Readers using usage-based reading found different and more unique defects, and they started to find defects earlier as well. These findings were confirmed in a replicated experiment.
In their experiment, didn’t use any active guidance in checklist-based reading., it is possible to incorporate an active guidance in checklist-based reading. In particular, readers were instructed to prioritize use cases according to their knowledge of the application domain, and use the prioritized use cases during reading.
The difference is that, in usage-based reading, use cases are prioritized by experts. They further compared the usage-based reading with the traditional checklist-based reading and its variant with an active guidance and concluded that
(1) checklist-based reading with an active guidance was significantly more effective and efficient than the traditional checklist-based reading in finding major defects, and
(2) usage-based reading was more effective and efficient than both types of checklist-based reading.
It takes time to develop detailed use cases. Use cases themselves shall be reviewed for accuracy and consistency. Thus use cases shall just have enough details to serve the purpose of usage-based reading. Thelin et al set forth to investigate if usage-based reading was still efficient and effective if use cases were prioritized, but without details except only their purposes.
Readers were actively developing use cases to fill in more details during reading. Not surprisingly, the researchers reported that reading with full use cases was more efficient and more effective than reading with incomplete use cases.
Reading with pre-developed use cases focused readers on detecting defects while reading with incomplete use cases forced readers to develop a detailed understanding of the documents, enabling them to detect different kinds of defects. Should an organization develop detailed use cases for usage-based reading beforehand or on the fly during reading?
Considering the total effort in developing use cases, the authors suggested not to spend time on developing use cases if these use cases are not used in the application development. The authors further suggested a hybrid approach, i.e., develop complete use cases if they are important and develop incomplete use cases if they are less important.
Erlansson et al. tested usage-based reading on detecting defects in requirements document was written in English. They compared usage-based reading to checklist-based reading, using graduate students as subjects. They reported that readers using usage-based reading didn’t detect more defects, even though they spent more time on the documents.
However, when the severity of defects was considered, usage-based reading detected much fewer minor defects and more moderate defects than checklist-based reading, and to their surprise, checklist-based reading seemed more effective in detecting critical defects.
The researchers added that their findings were not conclusive and they attributed the ineffectiveness of usage-based reading on requirements to several factors inherent to their experiments, including lack of training of the subjects. Information-overloading in usage-based reading could be another factor. Certainly, readers had to read through more documents such as, in this case, use cases.
Traceability-based reading is a family of reading techniques for inspecting high-level OO design in UML diagrams. The central feature of this family of reading techniques is to trace information between design documents (diagrams and textual descriptions) for design consistency and between design and requirements for design correctness and completeness, thus the name.
Types of Design Defects
Defects in software artifacts can generally be classified as omitted, ambiguous, inconsistent, incorrect, or extraneous information, which is consistent with the defect classification in requirements specification as discussed in defect-based reading. This classification can be tailored to design documents. The types of defects Traceability-based reading intended to detect include.
1. Omission: concept(s) from the requirements for a system are missing in the design document.
2. Incorrect fact: a design document has a misrepresentation of a concept described in the requirements for a system.
3. Inconsistency: a representation of a concept in one design document is inconsistent with a representation of the same concept in the same or another design document.
4. Ambiguity: a representation of a concept in the design is unclear and could cause a user of the design to misinterpret or misunderstand the meaning of the concept.
5. Extraneous information: the design includes unnecessary information.
High-Level OO Designs Using UML Diagrams
Software design is concerned with the description of real-world concepts that are part of the solution for the system envisioned in a requirements specification. High-level design deals with the problem description without considering constraints. That is, it takes functional requirements and maps them to, e.g., UML design diagrams.
This allows designers and developers to understand the problem first before attempting a solution. Low-level design deals with possible solutions. Low-level designs, which serve as a model for coding, depend on a high-level design and the non-functional requirements. We shall point out that what called high-level designs are sometimes referred to as analysis models.
Object-oriented development is widely practiced in the industry nowadays and UML diagrams are frequently used to document OO designs. Multiple UML diagrams are used to capture different views, different levels of abstraction, or different aspects of a system such as static or dynamic behaviors.
These diagrams might be drawn at different times by different designers using different viewpoints and abstractions. Consequently, it is hard to guarantee that these diagrams are consistent among themselves and that requirements are correctly and completely captured by these diagrams.
It is important to review and inspect high-level designs to ensure developers fully understand the problem before defining a solution as low-level designs (e.g., an activity diagram). It is more difficult and expensive to fix problems in low-level designs or code than fixing them in high-level designs.
Traceability-Based Reading Techniques
Travassos and colleagues developed a family of software reading techniques to detect defects in high-level OO designs captured in UML diagrams. They assume that the requirements specification document is available, along with use cases. However, these documents are not subject to review but used to aid the review of the design.
Designs are captured in UML diagrams and textual descriptions, including class diagrams, class descriptions, sequence diagrams, and state diagrams.
Traceability-based reading includes seven reading techniques. Each reading technique focuses readers on some aspects of the design. When all reading techniques are combined, a high degree of coverage of the design can be achieved. The seven reading techniques are organized into two groups, horizontal reading, and vertical reading. Relevant artifacts (design and/or requirement) are connected by lines with or without arrows and the lines are labeled with numbers 1 through 7.
The lines without arrows indicate a horizontal reading (readings 1-4) and the lines with arrows signal a vertical reading (readings 5-7). The dashed line separates lifecycle documents: above the dashed line are requirements description and use cases, which are generated in requirements definition phase; below the dashed line are high-level design artifacts created in the design phase and are subject to review.
Each reading technique has a detailed instruction on what to do and where to look for issues. Detected issues are logged as “discrepancy” instead of the defect and then the author of the design can decide if it is a defect, since the reader and author/designer may have different ideas about the design.
To facilitate the understanding of the reading techniques, let’s define some key terminologies
Functionality: A functionality describes the behavior of a system from the user’s point of view.
Service: A service is an atomic action performed by a system. One or multiple services are used to compose a system functionality. A service can be used in one or more functionalities.
Message: A message is the lowest level of behavior. It represents a communication between objects. It is typically shown on sequence diagrams and must be associated with object behaviors. One or more messages are used to compose a service.
Condition: A condition describes what must be true for a particular message to be executed.
Constraint: A nonfunctional constraint such as performance restricts the way a certain system functionality has to be implemented.
Functional requirements describe concepts and services a system has to fulfill. A use case describes important concepts and services a system provides for a user to accomplish a particular task and it typically describes execution paths through the system functionality. A class diagram describes classes of a system and how they are associated (inheritance, uses, composition, etc).
A set of class descriptions lists the classes of a system along with their attributes (properties, data members) and behaviors (methods). A state diagram describes internal states of an object and transitions between different states. State transitions are typically annotated with triggers. A sequence diagram describes classes and objects of a system and how they collaborate through sending and receiving messages to accomplish system services.
a. Horizontal Readings
Horizontal readings are designed to check if all of the design artifacts (class diagram, class description, state diagram, and sequence diagram) describe the same system. All diagrams are created in the same design phase, perhaps at different times. The consistency among them is the focus of horizontal reading.
Different diagrams contain complementary views to a system. Class diagrams and state diagrams capture the static behavior of the system, while sequence diagrams capture the dynamic one. All views are important, and together they allow developers to understand system behaviors and figure out how to accomplish them.
With reading techniques in this group, a reader mostly performs syntactic checking mechanically. There is not much application domain knowledge required. Instead, the reader’s development expertise plays a role here.
Reading 1: Sequence Diagram vs Class Diagrams
The purpose of this horizontal reading is to verify that classes and their relationships captured in a class diagram are consistent with the behaviors captured in a sequence diagram.
This reading is thus concerned with the static behavior of a system. To this end, a reader first verifies that classes and objects appearing in sequence diagrams also appear in class diagrams and then verifies that the relationships, behaviors, and conditions in class diagrams capture the services as described in sequence diagrams
For every sequence diagram, read and understand services the system provides and how these services should be implemented. In particular,
identify objects, classes, and actors in the sequence diagram;
identify the information exchanged between objects and determine if the information exchange represents messages or services (services are composed by messages); and
identify constraints on these messages and services and conditions when a message will be sent.
Identify and read class diagrams, and determine if the corresponding objects are documented correctly.
First verify for every object, class, and actor identified on sequence diagrams, there is a concrete class in a class diagram.
Then verify for every service or message identified on sequence diagrams, there is a class on a class diagram, which encapsulates the service or message, and the association between the sending and receiving object is also reflected on a class diagram. Note that in some cases, a reader may need to trace to the parent class of a class. The reader shall also determine if the series of messages is adequate to achieve the intended service.
Verify that constraints identified on sequence diagrams can be met based on the information on class diagrams. Those constraints can be the number of objects, permissible values of an attribute, dependencies between objects, timing constraints, etc.
Lastly, exercise some professional judgment if the design is sound, with respect to general design principles such as cohesion and decoupling as well as standards and guidelines.
If the reader cannot perform any steps or verify any points above, he or she shall raise a discrepancy. The author of the design will decide if a discrepancy is a defect or not.
Reading 2: State Diagrams vs Class Descriptions
The purpose of this horizontal reading is to verify that class descriptions capture the functionality specified in state diagrams
To perform this reading, a reader starts with state diagrams, and for each state diagram, he or she performs the following steps and verifies the associated points. If any step cannot be performed or any verification fails, the reader raises a discrepancy that will be analyzed by the author of the design.
Read and understand all possible and permissible states of an object, all actions that trigger state transitions, and how states and actions fit together.
Identify the class which the state diagram is for.
Start with the start state, trace all transitions and actions to trigger state transitions, and finish at the end state.
Read and understand the class or class hierarchy for which the state diagram models.
On the class description, identify the class or class hierarchy associated with the state diagram.
Identify how the class describes and encapsulates states identified from the state diagram above. Encapsulation may be done as an explicit or implicit attribute, a combination of attributes, or a class. If encapsulated as a class type, read the class and class hierarchy. Make sure all states are encapsulated in the same manner.
For each transition action identified on the state diagram, verify that there are behaviors to achieve that state transition. Behaviors can be implemented by the class or its parent classes in the inheritance chain. The transition action can be an event or constraint; thus look for an appropriate behavior.
Make sure the class as described in the class description captures the appropriate functionality, considering the system context and object behaviors and states.
Reading 3: Sequence Diagrams vs State Diagrams
State diagrams capture the static information and sequence diagrams capture the dynamic information. These different views allow developers to understand the system and objects from complementary viewpoints. However, these differences complicate reading and inspection, since a reader must combine them and identify possible discrepancies.
The purpose of this horizontal reading is to verify that every state transition for any object (read from state diagrams) can be achieved by messages sent and received by that object (read from sequence diagrams). Thus this reading is related to the dynamic aspects of a system.
To perform this reading, a reader starts with state diagrams and reads them one by one. For each state diagram, the reader performs the following steps and verifies the associated points and raises a discrepancy if any step cannot be performed or any verification fails.
Read and understand state diagrams. The relevant reading steps are the same as those for reading state diagrams in Reading 2.
Read sequence diagrams and understand how state transitions are achieved via messages sent and received by an object.
Identify the relevant sequence diagram(s) in which the object (modeled in the above state diagram) is used.
For each of these sequence diagrams, identify services and messages this object receives. On the state diagram, identify the states related to the service and the states leading to and from these states.
Map the messages on the sequence diagram to the state transitions on the state diagram. Make sure the mapping makes semantic sense.
Look for conditions and constraints of the mapped messages and verify those conditions and constraints are captured consistently on the state diagram.
Read through all state diagrams to ensure all state transitions are accounted for and have associated messages identified.
Identify any state transition on the state diagram that is not yet associated with object messages. If the transition is labeled with a constraint or event, check if there is a message, series of messages, or some action performed by an actor from outside of the system that can achieve the transition action.
When multiple messages are identified on the same sequence diagram for different state transitions of the same object, the reader shall also verify that the ordering of these messages reflects the topological order of the states (the state transition diagram is a directed graph).
Reading 4: Class Diagrams vs Class Descriptions
The purpose of this horizontal reading is to verify that class descriptions have all the information required by class diagrams and the class as described in class descriptions makes semantic sense. Thus this reading is related to static aspects of a system. It provided a detailed reading instruction. Here we sketch their main ideas.
To perform this reading, a reader starts with class diagrams and reads them one by one. For each class in a class diagram, the reader verifies the following points and raises a discrepancy if any verification fails:
The class has a class description.
The name and textual description of the class are meaningful and the description is at the right level of abstraction.
All attributes along with their types read from the class diagram are listed in and consistent with the class description. The class can encapsulate those attributes, and those attributes are implementable.
All behaviors and constraints read from the class diagram are present in and consistent with the class description. The class can encapsulate those behaviors, the behaviors are implementable, the constraints make sense for the class and are satisfiable, and the class has minimal dependencies on other classes.
The inheritance relationship is included in the class description, if applicable, and the class hierarchy is reasonable (i.e., it is an “is-a” relationship).
All class relationships (association, aggregation, or composition) are correctly annotated with regard to the multiplicity in the class diagram, the relationships make sense, the correct cardinalities are documented in the class description if important, and the relationship is represented by some class attribute, with a feasible type or data structure. Object roles and responsibilities are documented in the class description as well.
As the last step, the reader goes through all class descriptions and ensures that there is no class that is mentioned in the description but does not appear in any class diagrams. Otherwise, this should be reported as a discrepancy since it represents extraneous information.
b. Vertical Readings
Vertical readings are to check if design artifacts represent the right system as captured in the requirements specification and use cases, which are assumed to be correct for the design purpose. The artifacts involved here come from two different development phases, requirements definition, and design. Vertical reading thus prompts readers to trace and compare documents to ensure designs are correct and complete.
The artifacts represent different levels of abstraction and contain different levels of details. The design is considered a refinement of the requirements, and there is no single, direct mapping from requirements to the design. Vertical readings are thus complicated and a reader is required to abstract concepts from the design in order to match them in the requirements and use cases.
The purpose of this vertical reading is to verify concepts, and services described in the requirements are captured in class descriptions accurately and completely. This reading is thus concerned with the static behavior of a system. A reader starts with the requirements description to understand the functionality, as
The reader will then:
Read through the requirements one by one.
Identify nouns in the requirement that can be possible classes, objects, or attributes.
Also identify verbs or descriptions of actions that can be possible services or behaviors in the design.
Look for conditions, limitations, or constraints on these identified nouns and verbs.
With these identified nouns, verbs, conditions, and constraints, the reader next verifies these concepts are properly captured in class descriptions. For each and every action verb, find a related behavior or combination of behaviors in the class description. For each and every noun, find a related class in the class description.
The noun may be used as a class name, an instance name of the class, or an attribute name. If a class is matched, make sure its class description has sufficient information related to the concepts, the class encapsulates attributes and behaviors, and the conditions and constraints are described in the class description as well. If a class attribute is matched, make sure a feasible type is used for that attribute.
Lastly, make sure all nouns, verbs, conditions and constraints are reflected and not omitted in class descriptions.
Reading 6: Sequence Diagrams vs Use Cases
The purpose of this vertical reading is to verify that combinations of objects and messages sent among those objects as captured in sequence diagrams fulfill functionalities described by use cases. This reading is related to both static and dynamic aspects of a system.
To achieve the reading purpose, a reader reads and understands use cases first. He or she goes through each use case, identifies the functionality the use case describes and identifies important system concepts that are necessary to accomplish the functionality. Nouns in the use case described concepts of the system.
For those identified nouns, the reader further identifies verbs that describe actions applied to or by the nouns. Those identified verbs represent services the system provides. Constraints and conditions for those services, if any, shall also be identified. The reader also pays attention to the data or information exchanges in order to perform the actions.
Once having understood use cases, the reader identifies and reads related sequence diagrams, and identifies on these sequence diagrams the corresponding system objects, services, and data or information exchanges that were identified earlier in the use case. A correct set of sequence diagrams must be selected, which relies on a deep understanding of the system as well as traceability information.
As sequence diagrams are a refinement of the use case, the reader may need to identify messages and abstract out the services from these messages.
With these two sets of information identified on use cases and sequence diagrams, the reader next compares them semantically and verifies that they represent the same domain concepts.
The reader first verifies for every noun identified on the use case, there is a corresponding noun represented on sequence diagrams. For every unmatched noun identified on the sequence diagrams, the reader makes sure it is an attribute of some class by searching through class descriptions; otherwise, this could be an extraneous data.
The reader compares the services identified on sequence diagrams and on the use case. The reader shall focus on the ordering of the messages/services and data exchanged between objects.
Lastly the reader verifies that the constraints and conditions identified on the use case are observed on sequence diagrams. While reading and performing the above steps, the reader shall raise a discrepancy if any steps cannot be performed or any point fails the verification.
Reading 7: State Diagrams vs Requirements Descriptions and Use Cases
The purpose of this vertical reading is to verify that object states and events that trigger state transitions are correct and complete as described in the requirements and use cases. This reading is related to the dynamic behavior of a system.
To achieve the reading purpose, for each state diagram, a reader performs the following steps:
Read the state diagram to understand the object whose states are modeled.
Read the requirements description, compile a list of states for the object and complete the state adjacency matrix. To complete the task, the reader reads through the requirements description and focuses on where the concept related to the object is described, participates, or is affected. Among those places, the reader abstracts the different states.
To identify a state, look for the situations where the object behaves differently as a result of the change of an attribute or a group of attributes. Among all the states identified, determine the start state and end state. Draw an adjacency matrix table, where the row signals a “from” state and the column signals a “to” state.
For each pair of states (the order matters), mark if the state transition is allowed with the event and constraints if known or if the transition is forbidden.
Read use cases and further complete the adjacency matrix. Choose the use case descriptions where the concept is involved. If the state transition is allowed, but the event is missing, complete the event. For the state transitions not determined from the requirements, use the use cases to determine if the transitions are allowed with events or forbidden.
With those preparations, the reader is ready to examine the state diagram. The reader first checks if all states identified from requirements are represented on the state diagram. Be aware that those corresponding states may have different names. Also be mindful that two states might be merged. Conversely, make sure for every state shown on the state diagram, there is a state identified from the requirements.
Once the states from the state diagram and from the requirements are paired, compare the state transitions and their triggers. Again the comparison is a two-way check. Finally, make sure the constraints captured in the adjacency matrix are reflected on the state diagram. This is also a two-way check to guard against missing and extraneous constraints.
Like all other Traceability-based reading techniques, the reader shall log any discrepancy if any above steps cannot be performed or any verification fails.
The Traceability-based reading family includes seven reading techniques grouped into horizontal reading and vertical reading. In the horizontal reading, a reader compares design diagrams and a textual description. Since those documents represent the same level of abstraction, the comparison is relatively easy and syntactic checking is sufficient most of the time. With tools, the syntactic checking can be automated, which relieves the burdens on readers.
It is the semantic checking that proves to be challenging. This is apparent in the vertical reading group. For example, state names identified from the requirements may not be the same as those directly captured in the state diagrams. Even worse, two or more states can be merged in the design. The reader must understand the requirements and design in order to adequately deal with the situation.
As another example, the requirements and use cases talk about system services, but sequence diagrams use messages between objects. In order to properly trace between them, the reader must abstract services out of a group of messages.
The semantic checking is performed or required in the horizontal readings as well, although not as pervasive as in the vertical readings. The aforementioned situation of messages versus services also exists when reading sequence diagrams and state diagrams (in Reading 3, state diagrams use services, but sequence diagrams use messages).
In Reading 4, Class Diagrams vs Class Description, the reader is asked to decide if the class description makes semantic sense in terms of encapsulations of attributes and behaviors.
Because of this semantic checking, some level of subjectivity might be involved and a reader’s domain experiences play a significant role. That is why Travassos et al. call the detected issues discrepancies, not defects, since the authors may have different ideas or opinions than the reviewer.
Traceability-based reading defines seven individual reading techniques. For an industrial-scale project, many diagrams may have been created. It is thus a huge effort to conduct a full Traceability-based reading. Here we offer a few suggestions to tame the complexity.
If an incremental development process is used in your project, you can review artifacts gradually, as they are created incrementally. If this is not the case for your project, you can subdivide your system into units for inspection. Each inspection can be focused on some subset of system functionality in requirements or subset of conceptual entities in designs. As the mapping between the functionality and OO concepts is not trivial, there are issues with either approach.
The reading responsibilities can be distributed among team members, to reduce the workload of and to provide focus to individuals. The responsibilities can be divided along the dimension of reading techniques. One possibility is to ask one or more readers to perform the horizontal reading for consistency checking, ask one or more readers to focus on the vertical reading for traceability, and discuss and resolve the discrepancies at the end.
The other possibility is to ask one or more readers to focus on static views of a design (i.e., Readings 1, 2, 4, 5) and ask another one or more readers to focus on dynamic views (i.e., Readings 3, 7), with one in common for both static and dynamic views (i.e., Reading 6). The reading responsibilities can also be divided along the dimension of artifacts.
For example, reader A can focus on artifacts a, b, and c, while reader B can focus on artifacts x, y, and z. All readers use the seven reading techniques. Of course, the responsibilities can be divided using both reading techniques and artifacts.
If all diagrams and descriptions are available, the development team can use all seven reading techniques defined by Traceability-based reading. If some design artifacts do not exist, the team can just skip those reading techniques in which missing artifacts are used. Even if all artifacts exist, the development team can choose and apply a subset of reading techniques, depending on what attributes of the designs are important and what kinds of defects the team wants to detect.
In general, horizontal reading shall be performed before vertical reading, since we want to make sure the designs are consistent before validating them against the requirements. Specific situations may warrant the need to alter the reading order.
It used undergraduate students to evaluate the feasibility of Traceability-based reading. They reported that the reading techniques did allow subjects to detect defects, and a majority of the subjects thought the techniques were helpful. Vertical reading tended to find more defects in types of omission and incorrect facts, while horizontal reading tended to find more defects in types of ambiguity and inconsistency.
The research team continued to improve and refine the reading techniques by conducting observational studies and case studies. The reading techniques could be easily integrated into a development process, they indeed caught critical defects, and the effort was not prohibitively high when compared to other system tasks.
Traceability-based reading was successfully applied in industrial environments. Melo et al.reported a case study in which the reading techniques were applied in an industrial environment at Oracle Brazil, where a team of five people participated. Reading techniques allowed participants to find, on average, 35 defects (the most was 57 and the least 12).
The number of false-positive defects was very low (0.8 on average). The average time spent (individual reading and meeting) was 5.8 hours and the average time per defect was 15 minutes. Conradi et al.conducted an industrial experiment at Ericsson in Norway.
Traceability-based reading techniques were slightly tailored to their environment. They reported that Traceability-based reading techniques fit well in an incremental development process, and engineers employing the techniques found more defects (39 vs 25) than their existing reading techniques, which were a mixture of checklists and guidelines, while the cost efficiency was almost the same (1.28 vs 1.37 defects per hour).
Traceability-based reading seems to be complementary to their existing reading techniques as they found different types of defects, and engineers employing the new reading techniques found more defects of omission and inconsistency types. However, the new techniques need to be simplified with added questions and special guidelines for their project.
The ideas behind Traceability-based reading have been used in other reading techniques. One can clearly see the influence of Traceability-based reading in the perspective-based reading of design from an analyst’s viewpoint. The architecture reading techniques presented in the next section was also motivated by Traceability-based reading.
Software architecture is becoming a mature discipline. It is the blueprint for the software system development and has a far-reaching impact on organizations. Thus it is critical to review and gets it right early on (and review it regularly afterward).
The principal objectives of architecture reviews are to evaluate if the architecture can potentially deliver a system to fulfill the quality requirements and to identify potential risks. Babar and Gorton surveyed the current industry practice reported their own experiences on assessing industrial software architectures.
Architecting a system is to make fundamental, long-lasting, and hard-to-change decisions. These decisions can be implicit as made in an architect’s head or explicit as found in documentation and manifested as implemented in the system. Architecture review or evaluation is to assess all of these decisions. In this section we discuss architecture reading techniques, i.e., we are only concerned with reading and finding defects in software architecture documentation.
What Is a Software Architecture?
Software architecture is a widely used term now. There are many definitions that capture different aspects of software architecture, and there is no commonly accepted definition. We like the definition used by Clements et al: The software architecture of a system is the set of structures needed to reason about the system, which comprises software elements, relations among them, and properties of both.
Software architecture is important. There are many reasons, including inhibiting or enabling a system's quality attributes, reasoning about and managing changes, predicting system qualities even before it is built, enhancing communication among stakeholders, carrying early and fundamental design decisions, defining constraints on an implementation, influencing the organizational structure, enabling evolutionary prototyping, improving cost and schedule estimates, supplying a transferable and reusable model, allowing incorporation of independently developed components, restricting the vocabulary of design alternatives, and providing a basis for training.
A software architecture is shaped by many factors, and in turn, the architecture influences these factors. In addition to technical requirements, business, technical, project, and professional contexts influence the architecture. A software architecture is documented in a software architecture description that is both descriptive and prescriptive.
An architecture description typically has a textual description decorated with graphical diagrams, capturing module views, component-and-connector views, and allocation views using architecture patterns, styles, and framework.
We shall mention the differences between architecture and design. Architecture is design, but not all designs are architecture. The architecture is concerned with the fundamental structure of the system that delivers the quality attributes. These quality attributes are non-functional requirements and typically referred to as –ilities, such as availability, interoperability, modifiability, performance, security, testability, usability, and so on.
High-level design as discussed in the Traceability-based reading is a form of requirements analysis and expresses the requirements in analysis models, which are easy for designers to consume. The design is figuring out how to implement the functionalities within the software architecture constraints. Some authors dislike the term “detailed design,” as architecture design can occasionally be very detailed in some areas if the architects think they are important.
Traceability-Based Architecture Reading
The general concerns of architecture reading are thus to make sure different views are consistent in addition to complementary, and all views are correct with respect to and traceable to the requirements as well as other architecture drivers. The general ideas behind the Traceability-based reading appear to be applicable to architecture reading.
One horizontal reading and three vertical reading techniques are available for Traceability-based architecture reading:
The horizontal reading is to compare the information in one part of the architecture description to the other parts of the architecture description and ensure they are consistent. More specifically, the following pairwise comparisons between views are performed:
Logical structures (module views) vs communications patterns (component-and-connector views)
Logical structures vs physical structures (allocation views)
Communication patterns vs physical structures
The first vertical reading is to check if the logical decomposition captured in the module views is realistic, judged from the information in the requirements document.
The second vertical reading is to check if the communication patterns captured in the component-and-connector views are accurate based on the requirements document.
The third vertical reading is to compare the information about the stakeholders, architectural concerns, and rationale captured in the architecture description to the information in the requirements and ensure they are consistent.
Note that besides the requirements, other architectural drivers are not considered in the initial version of the architecture reading. An excerpt of the first vertical reading
1. Read through the requirements and compile a list of modules that implement the system.
2. For each module view in the architecture document, read through all relevant sections of the architecture document.
a. For each module in the module view, make sure there is a module identified from requirements in Step 1. If there is no match, log an issue as potentially extraneous information.
b. Conversely, for each module identified in Step 1, make sure there is a module view in architecture that contains that module. If there is no match, log an issue on potential missing information. If the match is not found in the current module view, decide if that module should have been included in the view.
c. Use the requirements as the guide to examine the relationship among modules. If the relationship does not make semantic sense (e.g., use relation, is a relation, part-of relation), log an issue as potential incorrect fact. This also decides whether the existence of a module in the module view makes sense or not, taking into account the intentions of the module view.
Carver and Lemon conducted a feasibility study of the Traceability-based architecture reading and compared it with checklist-based reading, using students as subjects. They concluded that, with checklist-based reading, subjects were more effective at finding defects of commission than defects of omission; with Traceability-based architecture reading, subjects were more effective at finding defects of omission than defects of commission.
Subjects reported that they preferred Traceability-based architecture reading over checklist-based reading, as the former helps readers focus and provides more detailed and structured guidance to read and inspect the whole document set. Traceability-based architecture reading appears to be feasible and easy to understand and use. The authors didn’t publish a follow-up study of the reading technique and there was no independent replication yet.
We shall be cautious on the omission defects, however. As the architecture description is meant to document the architecturally significant decisions, the architect may choose to leave some details to downstream designers.
In other words, the architect does not care whatsoever how a subpart of the system is designed and constructed. Thus a miss is not really a miss from the architect’s point of view. For that reason, readers log potential issues for architects to address.
Other Architecture Reading Techniques
As mentioned earlier, architecture reading is only a small portion of architecture review or evaluation and is concerned with reading and finding defects in architecture documentation. They presented a six-step procedure, and the fifth step is to perform the review, which is assisted with sets of questions.
They defined a template for a question set. Which set of questions to use depends on the purpose of an architecture review. During the lifecycle of a system, the architecture document shall be reviewed often and for different purposes, such as to check if it has captured the right stakeholders and their concerns in the concept phase or to check if it can support development in the development phase. The questions for a purposed review are organized for different stakeholders.
For example, software development managers, designers and implementers, integrators, testers, and quality assurance engineers are stakeholders when assessing whether the architecture document can support development. The concerns of these stakeholders are also provided to assist the architecture document review. The styles of the questions are sometimes similar to what one would expect in an active design review. One can clearly see some flavors of checklist-based and perspective-based readings in their approach.
We would be remiss if we didn’t mention the state of the practice for software architecture evaluation. Broadly speaking, there are two approaches to software architecture assessment, scenario-based and model-based. The very first architecture review technique is software architecture analysis method (SAAM).
The SAAM is a scenario-based method. Here a scenario is a brief description of some anticipated or desired use of a system. Essentially the SAAM develops, classifies, and prioritizes the scenarios and assesses how and if the scenarios can be mapped to the architecture.
The method was originally developed to assess an architecture for modifiability and superseded by a new general method, architecture tradeoff analysis method (ATAM). ATAM is applicable to assess almost any quality attribute, particularly the tradeoff among conflicting ones, but it is a very expensive method – it involves a big team for a few days.
A lightweight version has recently been proposed by the team at the Software Engineering Institute. Besides the general purpose methods, there are specific ones focusing on particular quality attributes, e.g., architecture-level modifiability analysis (ALMA), performance analysis of software architecture (PASA).
On the other hand, the model-based approach uses some formal models that can be automatically analyzed with tools. Architecture description languages are formal languages for describing and analyzing architectures. Interested readers can find more information in books on architecture evaluation
The objectives of software reading are for analysis or for construction. Almost all the reading techniques discussed in this book belong to the category of reading for analysis (defect detection). Here we discuss one reading technique, scope-based reading, of which the purpose is for construction.
Reading for construction is useful for software maintenance or new system development from reusable components, frameworks, and architectures. Scope-based reading was developed to enable readers to reuse both the design and implementation of OO application frameworks.
What Is an Application Framework?
An application framework—particularly an OO framework—is a reusable design and implementation of all or part of a system for a specific class of software, represented as a set of cooperating classes, including abstract ones, and the way their instances interact. It provides a skeleton application for a particular domain and regulates interactions among various constituting elements.
It typically has a set of hotspots where framework users can extend, customize, or plug in their own components. A framework serves to speed up application development by providing a foundation and a set of design and implementation elements that can be built upon and expanded as required.
From a software engineering perspective, it is desirable to develop an application framework to facilitate software development, since there are many advantages of framework-based development as claimed by proponents of this technology and supported by many successful industrial stories. Although the development of OO frameworks requires significant effort, the benefits generally justify the initial effort.
There are many different kinds of application frameworks. The development of OO application frameworks in general and domain-specific frameworks, in particular, has been a major industry thrust. There is a big body of literature on framework construction, but the literature on how to learn and use frameworks is scarce. The very steep learning curve of an application framework is notoriously known among software practitioners.
Scope-Based Reading Techniques
Scope-based reading is meant to help developers learn and use white-box OO frameworks. Here white-box means that the internal part of the framework, particularly the class hierarchy, is exposed to the users who typically derive subclasses to implement new functionalities for new applications. As the application domain becomes better understood, the framework becomes mature and it evolves from white-box toward black-box.
There are two effective ways to learn an OO application framework: by studying the class hierarchy of the framework, and by studying example applications. proposed two corresponding reading techniques: hierarchy-based reading and example-based reading, together known as scope-based reading.
These techniques give readers a different scope or breadth of knowledge about the framework, thus the name. Scope-based reading assumes an OO application framework is given, the reader wants to construct an application making use of the framework, he or she already has the object model and dynamic model of the application to be built, and the object model and dynamic model are not adapted to the application framework yet.
Hierarchy-based reading uses a stepwise approach. To help developers understand the functionality provided by the framework, it concentrates readers on the base or abstract classes first, then moves down the class hierarchy to the derived classes to find the most specific instantiation. It has the following steps:
1. Study the class hierarchy of the application framework. Determine which part of the framework architecture contains the functionalities that can be used or modified for the application under construction and identify base classes that can be used in the new application.
2. Identify classes related to the new application. For each class in the object model, find an appropriate class in the framework from which the class can be derived, and read and understand the framework class. If an appropriate class is not found, consider aggregating multiple framework classes to create a new class with the required attributes and operations.
3. Revise the object model and dynamic model for the application by using classes from the framework directly or classes composed from multiple framework classes, with details such as relevant attributes and operations.
4. Develop the application using the modified object models by deriving or aggregating framework classes wherever possible to reuse the attributes and operations provided by the framework. Write code as needed to extend the functionality and add new classes as needed to complete the new application.
Hierarchy-based reading has a steep learning curve. The reader has to understand how different pieces of the framework fit together before being productive. In practice, it is almost always supplemented with other reading techniques, such as the example-based reading technique discussed in the next section.
The main idea behind the example-based reading is to learn from examples. It assumes there is a collection of examples, illustrating the range of functionality and behavior provided by the framework. The examples can be provided by framework developers as applets or developed by others as real-world applications.
Further, assume the reader understands the functionality of the application he or she is constructing. Example-based reading is an iterative process since the new application will be built incrementally, feature by feature.
1. Decide which functionality should be added to the application under construction in the current increment. Search the collection of examples to find out which example provides the most complete coverage of that functionality. The reader can run each example, in turn, to check what features were implemented in the example.
2. Study the selected example in depth, but focus on the functionality being sought. By running the example, the reader constructs a set of use cases related to the sought-after functionality. For each use case, the reader traces the code, e.g., in the debug mode, to identify which objects/classes, attributes, and operations/methods are involved and how they cooperate to achieve the functionality.
It is necessary to read, understand, and document the relevant code along the way, including the object models and dynamic models related to the use cases.
3. Revise the object model and dynamic model for the application, based on the newly discovered or learned information.
4. Add the functionality to the application. The reader adds new classes modified from the example and augments existing ones with new attributes or methods. If the example does not implement the complete functionality, the reader writes additional code as needed. He or she also provides code to integrate the newly implemented functionality to the application.
5. Go back to Step 1 for the remaining functionalities, if the application is not complete yet.
During the above reading process, it is a good practice to record what is done, what decision is made, and the rationale behind the decision. For example, when selecting one particular example to study in depth, the reader can note which functionality is being sought, what are the candidate examples, and why a particular example is chosen, etc.
Example-based reading can quickly make the reader more productive. However, it is unlikely that the framework originator will or can provide a big set of examples to cover the complete features the framework provides and enables. It is also a concern, at least for a non-trivial application, whether the functionalities borrowed from different examples can coherently coexist in the final, finished application.
Shull et al. used software engineering students as subjects and conducted an exploratory experiment in an academic environment to test the effectiveness of hierarchy-based and example-based techniques.
They concluded, in their environment, that example-based techniques are well-suited to use by beginning learners and hierarchy-based techniques are not well-suited to use by beginners under a tight schedule. Example-based learning is a natural way to approach the learning of a complicated system.
The hierarchy-based technique is time-consuming and does not give developers an idea of where the starting place for implementation is or where they can look for certain functionality. The hierarchy-based technique would have been very effective if there has been sufficient documentation or the developers have been given more time.
The example-based technique is not without problems, however. Developers experienced difficulty in finding and extracting a small functionality embedded in much larger examples.
The effectiveness of the example-based technique depends on the breadth of functionality and other characteristics of example applications. Quite often the rationale of the design choice is missing from examples, which makes it hard for developers to reason about their choice. The example-based technique may limit the developers to explore beyond what examples provide.