Verification ensures that the system (software, hardware, documentation, and personnel) complies with an organization’s standards and processes, relying on review of non-executable methods.
Validation physically ensures that the system operates according to plan by executing the system functions through a series of tests that can be observed and evaluated.
Verification answers the question, “Did we build the right system?” while validation addresses, “Did we build the system right?”
Verification requires several types of reviews, including requirements reviews, design reviews, code walkthroughs, code inspections, and test reviews.
The system user should be involved in these reviews to find defects before they are built into the system. In the case of purchased systems, user input is needed to assure that the supplier makes the appropriate tests to eliminate defects.
Validation is accomplished simply by executing a real-life function. This includes unit testing, integration testing, system testing and user acceptance testing. In this rigorous testing is conducted to validate if the system meets the functional requirement.
The three most important skills that a system analysis should have are the same for any company. They should first and foremost have people skills. You have to be able to work with a variety of people and be able to work in teams. You should be an assertive person also. A good systems analysis should be able to take initiative and do things without being told. Also this person should have good reasoning and problem solving skills. These are all things that should be within the person naturally along with the actual computer skills necessary to analyze systems for a client. -are ability to work well with others,
-good communication skills,
-the ability to ask the right questions
Bidder Responsibility Determination:
To be determined responsible, a bidder must be successfully evaluated against the 7 following criteria:
1. Financial Resources.
The bidder must have adequate financial resources to perform the contract, or the ability to obtain them (see FAR 9.104-3(a)—Ability to Obtain Resources).
2. Performance Schedule.
The bidder must be able to comply with the performance schedule, required or proposed delivery, taking into consideration all existing commercial and governmental business commitments.
3. Performance Record.
The bidder must have have a satisfactory performance history, if any (see FAR 9.104-3(b)—Satisfactory Performance Record and Experience Certificate). Nevertheless, a prospective contractor shall not be determined responsible or non-responsible solely because of a lack of relevant performance history, except when specified in a standard for special acquisitions.
4. Integrity and Ethics.
The bidder must have a satisfactory record of integrity and business ethics including satisfactory compliance with laws related to taxes, labor and employment, environment, antitrust, and consumer protection (see FAR 9.406-2—Causes for debarment and FAR 9.407-2—Causes for suspension).
5. Organization and Skills.
The bidder must have the necessary organization and skills, experience, accounting and operational controls, and technical skills, or the ability to obtain them (see FAR 9.104-3(a)—Ability to Obtain Resources).
6. Equipment and Facilities.
The bidder must have the necessary technical equipment and facilities for production or construction, or ability to obtain them (see FAR 9.104-3(a)—Ability to Obtain Resources); and
7. Other Qualification.
The bidder must be otherwise qualified and eligible to receive an award under applicable laws and regulations.
Systems Engineering V Model
The system life cycle
The system life cycle has seven phases: (1) discovering system requirements, (2) investigating alternatives, (3) full-scale engineering design, (4) implementation, (5) integration and test, (6) operation, maintenance and Assessment and (7) retirement, disposal and replacement. However, the system life cycle is different for different industries, products and customers.
State the problem
The problem statement starts with a description of the top-level function that the system must perform or the deficiency that must be ameliorated. It includes system requirements stated in terms of what must be done, not how to do it. It might be composed in words or as a model. Inputs come from end users, operators, bill payers, owners, regulatory agencies, victims, sponsors, Marketing, Manufacturing, etc. These are called stakeholders. In a modern business environment, the problem statement starts with a reason for change followed by vision and mission statements for the company.
Understand customer needs
Customers seldom know what they want or need. Systems Engineers must enter the customer’s environment and find out how the customer will use the system. Talking to your customer’s customer and your supplier’s supplier can be very useful. Frameworks, such as the Zachman framework or the DoDAF, are useful for seeing how the system fits into the customer’s enterprise.
Discover system requirements
There are two types of system requirements: mandatory and tradeoff Mandatory requirements insure that the system satisfies the customer’s operational need, and must be passed or failed, there is no middle ground. The tradeoff requirements are evaluated to determine the preferred designs, and should state conditions that would make the customer happier.
Verify and validate requirements
Investigate alternatives
Alternative designs are evaluated based on performance, cost, schedule and risk criteria. This analysis should be redone whenever more data are available.
Define quantitative measures
Performance and cost criteria show how well the system satisfies its requirements, e.g., In this test the car accelerated from 0 to 60 in 6.5 seconds. Technical performance measures (TPM’s) are made during the design and manufacturing process to evaluate the likelihood of satisfying the system requirements.
Model the system
Models will be developed for most alternative designs. Many types of system models are used, such as block diagrams, functional flow diagrams, object-oriented models, computer simulations.
Design the system
The overall system must be partitioned into subsystems, subsystems must be partitioned into assemblies, etc. Reusability should be considered in creating subsystems. For new designs, subsystems should be created so that they can be reused in future products. For redesign, subsystems should be created to maximize the use of existing, particularly commercially available, products. Systems engineers must also decide whether to make or buy the subsystems, first trying to use commercially available subsystems. If nothing satisfies all the requirements, then modification of an existing subsystem should be considered. If this proves unsatisfactory, then some subsystems will have to be designed in-house. Flexibility is more important than optimality. Hardware, software and bioware must be considered. Bioware (or wetware) means humans and other biological organisms that are a part of the system. For example, in designing a race track the horses or dogs are a part of the bioware.
Create sequence diagrams
Define system architecture
Some choices that have to be made: (1) object-oriented design, structured analysis, or functional decomposition, (2) distributed or centralized computing, (3) commercial off the shelf (CoTS) or custom designed.
Functional analysis
Systems engineers do functional analysis on new systems (1) to map functions to physical components, thereby ensuring that each function has an acknowledged owner, (2) to map functions to system requirements, and (3) to ensure that all necessary tasks are listed and that no unnecessary tasks are requested. This list becomes the basis for the work breakdown structure. A work breakdown structure (WBS) breaks a project into smaller, more manageable components.
Sensitivity analyses
Sensitivity analyses can be used to point out the requirements and parameters that have the biggest effects on cost, schedule and performance. They are used to help allocate resources.
Assess and manage risk
There are two types of risk: risk of project failure (due to cost overruns, time overruns or failure to meet performance specifications) and risk of harm (usually called personnel safety). A failure modes and effects analysis and risk mitigation must be performed. Project risk can be reduced by supervising quality and timely delivery of purchased items.
Reliability analysis
Major failure modes must be analyzed for probability of occurrence and severity of occurrence.
Integrate system components
Integration means bringing things together so they work as a whole. System integration means bringing subsystems together to produce the desired result and ensure that the subsystems will interact to satisfy the customer’s needs. End users and engineers need to be taught to use the system with courses, manuals and training on the prototypes.
Design and manage interfaces
Interfaces between subsystems and interfaces between the main system and the external world must be designed. Well-designed subsystems send finished products to other subsystems. When designing subsystems and their interfaces be sure to consider reuse.
Launch the system
Launching the system means doing what the system was intended to do, e.g. running the system and producing outputs.
Configuration management
Configuration management (also called modification management) ensures that any changes in requirements, design or implementation are controlled, carefully identified, and accurately recorded. All stakeholders should have an opportunity to comment on proposed changes. Decisions to adopt a change must be captured in a baseline database. Baselines can only be changed at specified points in the life cycle. The phrase requirements tracking is now being used for an important subset of configuration management.
Project management
Project management is the planning, organizing, directing, and controlling of company resources to meet specific goals and objectives within time, within cost and at the desired performance level. Project management creates the work breakdown structure, which provides structure for guiding team assignments and cost and tracking control.
Documentation
All of these Systems Engineering activities must be documented in a common repository, often called the Engineering Notebook. The stored information should be location, platform, and display independent: which means any person on any computer using any tool should be able to operate on the fundamental data. Assumptions, results of tradeoff studies and the reasons for making critical decisions should be recorded. These documents should be alive and growing. For example, at the end of the system life cycle there should be an accurate model of the existing system to help with retirement.
Lead teams
Complex systems cannot be designed by one person. Consequently engineers work on Integrated Product Development Teams (IPDTs). These teams are interdisciplinary with members from Business, Engineering, Manufacturing, Testing, etc. IPDTs are often led by Systems Engineers.
Assess Performance
During the operation and maintenance phase of the system life cycle the performance of the system must be measured. Initially these measurements will be used to verify that the system is in compliance with its requirements. Later they will be used to detect deterioration and initiate maintenance.
Prescribe tests
Early in the system life cycle Systems Engineering should describe the tests that will be used to prove compliance of the final system with its requirements. However, most testing should be performed by built-in self-test equipment. These self-tests should be used for initial testing, post-installation testing, power-up diagnostics, field service and depot repair. The recipient of each test result and the action to be taken if the system passes or fails each test must be stated.
Conduct reviews
Systems Engineering should ensure that the appropriate reviews are conducted and documented. The following set is common: Mission Concept Review, System Requirements Review (SRR), System Definition Review, Preliminary Design Review (PDR), Critical Design Review (CDR), Production Readiness Review (PRR), and System Test. Full-scale engineering design begins after the Preliminary Design Review. Manufacturing begins after the Critical Design Review.
Total system test
The system that is finally built must be tested to see (1) that it satisfies the mandatory requirements, and (2) how well it satisfies the tradeoff requirements.
Re-Assessment
Re-Assessment is arguably the most important task of Systems Engineering. For centuries engineers have used feedback to control systems and improve performance. It is one of the most fundamental engineering tools. Re-Assessment means observing outputs and using this information to modify the system inputs, the product or the process. Re-Assessment should be a continual process with many parallel loops. Everyone should continually re-evaluate the system looking for ways to improve quality. Tools used in this process include basic systems engineering, and the quality engineering techniques presented by, for example, Deming and Taguchi. Deming (1982); Bicknell and Bicknell (1994); Latzko and Saunders (1995). Near the end of the project, engineers should write a Lessons Learned document. These lessons learned should not be edited by management, because management could trivialize what they do not understand or omit management mistakes.
Categories of Systems Engineers
Many companies divide their Systems Engineers into three categories according to their major workflows: requirements definition, architectural design and testing and verification.
Creating Systems Engineers
The traditional method of creating Systems Engineers was to select well-organized engineers with lots of common sense and let them acquire 30 years of diverse engineering experience. But recently these traditional Systems Engineers have written books and standards that explain what they do and how they do it. So now that the tools, concepts and procedures have been formalized, in four years of undergraduate education we can teach Systems Engineers who will have performance levels 50% that of traditional Senior Systems Engineers. Ten years of systems engineering experience will improve performance to 80% and another ten years will increase it to 100%.
