{"text_format":"1","workbook":"xlsx","n_sheets":11,"sheets":["Introduction","Instructions","V 2017 Notes","Database","Criteria","MRL Users Guide","Definitions","List of Acronyms","MRL Printable Matrix_V2020","Questionnaire","Summary Pivot"],"not_rendered":{"charts":0,"images":2,"macros":1,"external_links":3,"defined_names":36,"comments":1,"pivot_tables":1},"unopened":[],"bounds":{"document_bytes":16777216,"sheet_cells":100000,"rows_first":2000,"rows_last":500,"rows_between":2000,"halvings":0,"cell_chars":16384,"cell_head_chars":4096},"cells_cut":0,"cells_raw":0,"read":"whole","date_system":"1900"}

{"sheet":"Introduction","index":0,"n_rows":30,"n_cols":3,"rows_kept":30,"cut":false,"hidden":false,"merged":[[0,1,0,2],[1,1,1,2],[9,1,9,2]]}
[null,"MRL Users Guide "]
[null,"Introduction"]
[null,"This MRL Users Guide is based on guidance from DoDI 5000.02T, Change 7, dated 21 April 2020 and consists of six worksheets:"]
[null,null,"(1) This first worksheet is the MRL Users Guide Introduction."]
[null,null,"(2) The second worksheet is the digital MRL Users Guide. This Guide has the capability to display detailed information about the MRL or Product Life Cycle by simply clicking on a given cell or icon for which information is desired. "]
[null,null,"(3) The third worksheet is a definitions list of terms typically used in the acquisition and manufacturing readiness assessment process."]
[null,null,"(4) The fourth worksheet is a list of acronyms commonly used in the acquisition process."]
[null,null,"(5) The fifth worksheet contains the standard 11' x 17\" MRL Printable Matrix for those who wish to view or print the entire matrix as hard copy. (Note: Your printer must first be formatted for the desired size paper and the size adjusted appropriately.) "]
[null,null,"(6) The sixth worksheet  is the Questionnaire which contains a complete list of questions (criteria), derived from the MRL Matrix, to be used in assessments of manufacturing readiness.  This list of questions is NOT intended to be answered in its entirety! It should be tailored to the system, subsystem, or component being assessed and be limited to questions focused on the target MRL or one level lower. Make sure filters are turned \"ON\" under \"Data\" and \"Filter\"."]
[null,"How to use the MRL Users Guide "]
[null,"Layout of the Users Guide"]
[null,null," The cells down Column A provide information about the specific threads that are traced in that row of the matrix. "]
[null,null," The cells and icons in Rows 2 through 6 display information about the phases of the Product Life Cycle, Acquisition Reviews, Acquisition Milestone descriptions, MRL and TRL definitions and background information for that stage of the product life-cycle. "]
[null,null,"The Thread designation (e.g., A1, B2, etc.) and MRL Number (1 through 10) will appear at the top."]
[null,null,"The associated text (criteria) of the cell from the MRL Matrix will appear in the next block. These are for reference to let you know which cell you are viewing."]
[null,"To activate the Users Guide"]
[null,null,"Check the  \"Enable content\" and \"Enable pop-up text\" boxes at the top.  "]
[null,null,"Click on a cell to display information about that cell or topic. If the cell contains more information than can be displayed on a single screen, you can scroll up or down by using the bar at the right hand side of the window or the up and down arrow keys on your keyboard.  "]
[null,null,"Note: If your screen resolution does not permit you to view the entire data sheet you also have the option to display the entire form as a tabbed page (the background will not be visible if you choose this option)."]
[null,"MRL Matrix Information Pop Up Information"]
[null,null,"Purpose- The intent for doing the assessment for this particular sub-thread at this point in the life-cycle and the reason for doing the assessment of this particular thread at this point, i.e., what requirements/documents/procedures drive the assessment? "]
[null,null,"Sources of Information-  This is where data can be collected for a particular MRL Assessment at that stage of the product life-cycle."]
[null,null,"Questions (also called \"criteria\")-  These are directly derived from the text of the MRL Matrix from the latest revised Version 11.5 of the MRL Questionnaire.      "]
[null,null,"Additional Considerations-   Sometimes from past experience, services or industry have optional questions they may want to ask for specific threads or sub-threads at specific times in the life-cycle.  "]
[null,null,"Lessons Learned-  These are particular lessons derived from past experience of personnel doing risk assessments in this particular sub-thread at this specific point in the product life-cycle. These may also change as people gain more experience doing assessments of manufacturing risk and readiness."]
[null,"To Print the Users Guide"]
[null,null,"Simply click on the box at the top of the Guide that says \"View form as a tabbed page.\" When you are in a cell that displays data, you have the option to print the contents of the window or simply close it."]
[null,null,"Press the print button on the bottom left hand side of the window, another window will pop up to confirm your choice to print in case you wish to cancel printing. "]
[null,null,"If you want to view the contents of adjacent windows you can use the buttons on the bottom left-hand side of the window to page up, down, left, or right. "]
[null,null,"Note:  If you close a cell you will have to click out, or select another cell, before you can view that cell again. When they become available, additional information and URL hot-links will be provided to source documents (i.e., DoDI 5000.02, the Defense Acquisition Guide, MRL Deskbook, etc.) where more information may be found about the topic in the cell you are viewing.      "]

{"sheet":"Instructions","index":1,"n_rows":7,"n_cols":1,"rows_kept":7,"cut":false,"hidden":true,"merged":[]}
["MRL Users Guide Instructions"]
["This MRL Users Guide is based on guidance from DoDI 5000.02 dated 7 January 2015 and consists of SEVEN worksheets: (1) This first worksheet is the MRL Users Guide Instructions. (2) The second worksheet is the digital Users Guide. This Guide has the capability to display detailed information about the MRL or Product Life Cycle by simply clicking on a given cell or icon for which information is desired.  The cells down Column A provide information about the specific threads that are traced in that row of the matrix.  The cells and icons in Rows 2 through 6 display information about the phases of the Product Life Cycle, Acquisition Reviews, Acquisition Milestone descriptions, MRL and TRL definitions and background information for that stage of the product life-cycle.  (3) The third worksheet is a definition list of terms typically used in the acquisition and manufacturing readiness assessment process. (4) The fourth worksheet is a list of acronyms commonly used in the acquisition process. (5) The fifth worksheet contains the standard 11' x 17\" MRL Matrix for those who wish to view or print the entire matrix as hard copy. (Note: Your printer must first be formatted for the desired size paper and the size adjusted appropriately.) (6) The sixth worksheet contains a complete list of questions (criteria), derived from the MRL Matrix, to be used in assessments of manufacturing readiness.  This list of questions is NOT intended to be answered in its entirety! It should be tailored to the system, subsystem, or component being assessed and be limited to questions focused on the target MRL or one level lower. Make sure filters are turned \"ON\" under \"Data\" and \"Filter\".  (7) The seventh worksheet is a pivot table roll-up of data entered into the sixth worksheet.  You may filter for specific MRLs using the drop-down filters.                                                                                                                                 "]
["To activate the Users Guide, first be sure that the \"Enable content\" and \"Enable pop-up text\" boxes at the top are checked, then click on a cell to display information about that cell or topic.  If the cell contains more information than can be displayed on a single screen, you can scroll up or down by using the bar at the right hand side of the window or the up and down arrow keys on your keyboard.  If your screen resolution does not permit you to view the entire data sheet you also have the option to display the entire form as a tabbed page (the background will not be visible if you choose this option). Simply click on the box at the top of the Guide that says \"View form as a tabbed page.\" When you are in a cell that displays data, you have the option to print the contents of the window or simply close it.  If you press the print button on the bottom left hand side of the window, another window will pop up to confirm your choice to print in case you wish to cancel printing.  If you want to view the contents of adjacent windows you can use the buttons on the bottom left-hand side of the window to page up, down, left, or right. Note: If you close a cell you will have to click out, or select another cell, before you can view that cell again. When they become available, additional information and URL hot-links will be provided to source documents (i.e., DoDI 5000.02, the Defense Acquisition Guide, MRL Deskbook, etc.) where more information may be found about the topic in the cell you are viewing.                                                                                                                                                                                                                                                                                         "]
["When specific cells in the main body of the MRL Matrix are selected, they will display a \"Mega-Data Sheet\" with the following general information:  First, the Thread designation (e.g., A1, B2, etc.) and MRL Number (1 through 10) will appear at the top, then the associated text (criteria) of the cell from the MRL Matrix will appear in the next block. These are for reference to let you know which cell you are viewing.  The main body of the Mega-Data Sheet contains the following information:                                                                                                                                                                                                                                                                               \nPurpose:  The intent for doing the assessment for this particular sub-thread at this point in the life-cycle and the reason for doing the assessment of this particular thread at this point, i.e., what requirements/documents/procedures drive the assessment? \nSources of Information: This is where data can be collected for a particular MRL Assessment at that stage of the product life-cycle.\nQuestions (also called \"criteria\"): These are directly derived from the text of the MRL Matrix from the latest revised Version 11.5 of the MRL Questionnaire.                                                                                                                                                                       Additional Considerations:  Sometimes from past experience, services or industry have optional questions they may want to ask for specific threads or sub-threads at specific times in the life-cycle.  If so, these will be included in the mega-data sheet. This part of the Users Guide may change significantly over time.  \nLessons Learned: These are particular lessons derived from past experience of personnel doing risk assessments in this particular sub-thread at this specific point in the product life-cycle. These may also change as people gain more experience doing assessments of manufacturing risk and readiness."]
["NOTES: V11.3.13 and following fix bugs occurring when you run this application on MsExcelTM 2013. When you open the MRL Users Guide Worksheet and click on a cell, you may see only the data from that cell displayed with the MRL Matrix blanked out in the background. This mode will enable you to view the entire data sheet and its control buttons. If you wish to view the matrix in the background, uncheck the box at the top of the form that says, \"View form as a tabbed page.\" If you do this, you may have to change your screen resolution to view the entire data sheet with its control buttons. If you wish to copy these instructions, you may find it easier to select each row of the worksheet separately and use the \"copy\" utility to copy it to an MsWord.doc."]
["NOTES: V.2016, added changes IAW with Criteria Matrix 2016, single and sole source definitions, and fixed close button issues.   Added horizontal-based questionnaire and pivot table version of MRL Questionnaire.  "]
["NOTES: V.2017, added changes IAW with Criteria Matrix 2016,hard artificats and objective evidence MRL 4-10.  Removed pivot table version of MRL Questionnaire.  "]

{"sheet":"V 2017 Notes","index":2,"n_rows":4,"n_cols":1,"rows_kept":4,"cut":false,"hidden":true,"merged":[]}
[]
["NOTES: V11.3.13 and following fix bugs occurring when you run this application on MsExcelTM 2013. When you open the MRL Users Guide Worksheet and click on a cell, you may see only the data from that cell displayed with the MRL Matrix blanked out in the background. This mode will enable you to view the entire data sheet and its control buttons. If you wish to view the matrix in the background, uncheck the box at the top of the form that says, \"View form as a tabbed page.\" If you do this, you may have to change your screen resolution to view the entire data sheet with its control buttons. If you wish to copy these instructions, you may find it easier to select each row of the worksheet separately and use the \"copy\" utility to copy it to an MsWord.doc."]
["NOTES: V.2016, added changes IAW with Criteria Matrix 2016, single and sole source definitions, and fixed close button issues.   Added horizontal-based questionnaire and pivot table version of MRL Questionnaire.  "]
["NOTES: V.2017, added changes IAW with Criteria Matrix 2016,hard artificats and objective evidence MRL 4-10.  Removed pivot table version of MRL Questionnaire.  "]

{"sheet":"Database","index":3,"n_rows":341,"n_cols":6,"rows_kept":341,"cut":false,"hidden":true,"merged":[]}
["CellRef","Row","MatrixText","ReferenceText","LinkText1","LinkAddress1"]
["$A$1",2,"Version 2020\n"]
["$B$1",3,"User's Guide v.2020"]
["$C$1",4]
["$D$1",5]
["$E$1",6]
["$F$1",7]
["$G$1",8]
["$H$1",9]
["$I$1",10]
["$J$1",11]
["$K$1",12]
["$L$1",13]
["$A$2",14,"DoD Manufacturing Readiness Levels (MRLs)"]
["$B$2",15]
["$C$2",16]
["$D$2",17]
["$E$2",18]
["$F$2",19]
["$G$2",20]
["$H$2",21]
["$I$2",22]
["$J$2",23]
["$K$2",24]
["$L$2",25]
["$A$3",26,"Acquisition Phase"]
["$B$3",27]
["$C$3",28,"Pre Materiel Solution Analysis (Pre MSA)","     Prior to initiation of any warfighter capability need, pre-acquisition efforts within DoD focus on the exploration of new science and technologies that become the building blocks for future capabilities, and on gaining understanding and knowledge of how new concepts might be implemented by the warfighter.  These two categories of effort complement each other to generate the new capabilities the warfighter needs to be successful in the future.  \nThe development of new science and technology through investments in basic and early applied research generate the foundation of new technology that goes into future systems.  While efforts are predominantly knowledge-building and early bench testing of science and technology, the technologist working on these efforts needs to apply an understanding of potential warfighter uses of the new technologies with a broad awareness of all facets of development ranging from capability improvement potential to affordability, manufacturability and even operator integration.\n     Concept exploration and experimentation investigates how new and even past approaches to employing operational methods and tactics can create greater capability with and without employment of technology, both new and existing.  Experimentation is used to demonstrate utility success and provide inputs to new capability requirements that form Materiel Capability Decisions leading to new program initiation.\n     Combined, these two approaches produce the basis for acquisition program initiation.  Awareness and consideration of how new technology will be matured, implemented and sustained over a program’s complete lifecycle needs a foundation from infancy to be most successful.\n     One of the key factors for this early exploration to eventually providing the warfighter a successful capability is to start addressing potential constraints (e.g., cost, performance, manufacturing, etc.) as part of the solution. The S&T and acquisition communities have gated processes as technology matures and moves to higher levels of funding and integration.  Developing a solid understanding of these potential constraints is essential to providing the warfighter with capability needed in a timely and cost-effective manner.  In order to make sound decisions at each gate, it is imperative that technology developers acquire an understanding of these constraints in parallel with the development of the technology they will deploy."]
["$D$3",29]
["$E$3",30]
["$F$3",31,"Materiel Solution Analysis (MSA)  ","     The Materiel Development Decision marks the start of the MSA Phase. This presents the first substantial opportunity to influence systems design by balancing technology opportunities, schedule constraints, funding availability, system performance parameters, and manufacturing feasibility. The technical approach for system development should be driven by knowledge of the manufacturing maturity and risk of the various technologies under consideration as well as their associated performance maturity. Two systems engineering reviews, the Alternative Systems Review (ASR) and the Initial Technical Review (ITR), should be conducted during MSA.\n     During the MSA Phase, an assessment of manufacturing readiness is conducted for each competing materiel solution being examined in the AoA with special emphasis on the proposed materiel solution to analyze feasibility from a manufacturing perspective and determine manufacturing resources needed. It is in effect a manufacturing feasibility assessment. Sources of data may include technology and mission area plans and roadmaps, market research, and early evaluations of technology maturity.\n    Concept development and refinement occur before Milestone A, and further technology development to flesh out the concept occurs before Milestone B. Pre-Milestone A efforts involve translating the needs of the user into clearly stated key performance parameters (KPPs) and evolving the system concept and preliminary concept of operations (CONOPS) that satisfy these needs. Only after Milestone B does a program become an enterprise with dedicated funding behind it.\n     About three-quarters of the total system life cycle costs are influenced by decisions made before the end of the concept refinement phase at Milestone A, while about three-quarters of life cycle funds are not actually spent until after Milestone C. This means that while engineering and development is necessary during the entire acquisition cycle, the application of high-quality system analysis, concept definition and early systems engineering to decisions made in the pre-Milestone A period is critical to avoiding (or at least minimizing) cost and schedule overruns later in the program.\n\nPurpose of the Materiel Solution Analysis Phase\n     The purpose of this phase is to conduct the analysis and other activities needed to choose the concept for the product that will be acquired, to begin translating validated capability gaps into system-specific requirements including the Key Performance Parameters (KPPs) and Key System Attributes (KSAs), and to conduct planning to support a decision on the acquisition strategy for the product.  AoA solutions, key trades between cost and performance, affordability analysis, risk analysis, and planning for risk mitigation are key activities in this phase.\n\nPhase Description\n     Minimum funding required for this phase is normally that needed to analyze and select an alternative for materiel development, and to complete the activities necessary to support a decision to proceed to the next phase; technology development and concept analysis and design efforts may also be funded in this phase.\n     The validated ICD and the AoA Study Plan will guide the AoA and Materiel Solution Analysis Phase activity.  The analysis will be conducted in accordance with the procedures in Enclosure 9 of DoDI 5000.02, and focus on identification and analysis of alternatives; measures of effectiveness; key trades between cost and capability; total life cycle cost, including sustainment; schedule; concepts of operations; and overall risk. The AoA will inform and be informed by affordability analysis, cost analysis, sustainment considerations, early systems engineering analyses, threat projections, and market research.\n     Prior to the completion of this phase, the DoD Component combat developer will prepare an Operational Mode Summary/Mission Profile (OMS/MP) that will include the operational tasks, events, durations, frequency, operating conditions and environment in which the recommended materiel solution is to perform each mission and each phase of a mission. The OMS/MP will be provided to the Program Manager and will inform development of the plans for the next phase including: acquisition strategy, test planning, and capability requirements trades.  It will be provided to industry as an attachment for the next acquisition phase RFP.\n     This phase ends when a DoD Component has completed the necessary analysis and the activities necessary to support a decision to proceed to the next decision point and desired phase in the acquisition process. The next phase can be Technology Maturation and Risk Reduction (TMRR), EMD, or Production and Deployment, depending on the actions needed to mature the product being acquired. Each of these phases has associated decision points to authorize entry: Milestone A, Development RFP Release and Milestone B, or Milestone C. \n\nProgram Office Establishment and Next Phase Preparation\n     During the Materiel Solution Analysis Phase, the CAE will select a Program Manager and establish a Program Office to complete the necessary actions associated with planning the acquisition program with emphasis on the next phase. Prior to preparation and release of a final RFP for the planned next phase, the Program Manager should complete and submit the Acquisition Strategy and obtain MDA approval. An approved Acquisition Strategy will inform development of the final RFPs for the next phase of the program."]
["$G$3",32,"Technology Maturation and Risk Reduction (TMRR)","     The Milestone A decision point marks the entry into the TMRR Phase of acquisition. TMRR is a focused effort to mature, prototype, and demonstrate technologies in a relevant environment. The purpose of this phase is to reduce technology risk and to determine the appropriate set of product technologies and manufacturing capabilities to be integrated into a full system. Three major systems engineering reviews are normally conducted during this phase, the System Requirement Review (SRR), the System Functional Review (SFR), and the Preliminary Design Review (PDR).\n     A Technology Readiness Assessment (TRA) is completed just prior to Milestone B. When feasible, this TRA should be closely coordinated with the assessment of manufacturing readiness conducted at that time. Manufacturing subject matter experts should participate in the TRA process.\n     TMRR ends in a decision on the preferred system concept that provides a low risk entry into EMD. Just as it is expected that technologies will be brought to TRL 6 or better by the end of this phase, manufacturing capabilities should also be brought to at least MRL 6.\nAt the end of the TMRR Phase, an assessment of manufacturing readiness is conducted to baseline needed industrial capabilities and identify remaining required investments for every competing design or prototype that has conducted a PDR at the full system level. It is in effect a manufacturing capability assessment. Sources of data may include the results of SRR, SFR and incremental PDRs, ICAs, program risk management plans, and the TRA.\n     While it is not expected that contractors would have a complete factory and supply chain established this early in a program, key knowledge must be obtained on 3-6 critical manufacturing processes, production scale-up efforts, and potential supply chain issues. The results of the assessment of manufacturing readiness performed during the MSA Phase should be used as a baseline reference for this activity. It is possible that some technology development activities were not assessed during the MSA Phase. In that case, it is a best practice to conduct an assessment early in the TMRR Phase to establish a baseline if manufacturing risk is great enough to warrant the effort. Technology Transition Agreements should be used to manage the transition process from a manufacturability and producibility standpoint. Technologies identified to have a maturity level less than MRL 4 at the start of this phase require special attention for maturation and risk mitigation in order to bring them to MRL 6 by Milestone B."]
["$H$3",33]
["$I$3",34,"Engineering & Mfg Development (EMD)","     Milestone B determines whether a formal acquisition program will be launched and marks the entry point into the EMD Phase. This phase completes the development of a system, leverages design considerations, completes full system integration, develops affordable and executable manufacturing processes, and completes system fabrication, test and evaluation. The systems engineering reviews normally conducted during this phase are the CDR, the Test Readiness Review (TRR), the System Verification Review (SVR) (Functional Configuration Audit) and the PRR.\n     From a manufacturing perspective, the purpose of the EMD phase is to ready the acquisition program for production by implementing manufacturing risk reduction activities that are reflected in the acquisition strategy. The basic manufacturing planning that was developed in the previous phase should be detailed in EMD and significant program emphasis should be placed on bringing all hardware to the target MRL prior to the decision point at which this phase ends—the authorization to enter LRIP or FRP for non-major systems that do not require LRIP. MRL 8 is the target for LRIP and MRL 9 is the target for FRP; these targets should be reflected in the acquisition program baseline.\n     During EMD, assessments of manufacturing readiness are conducted to identify remaining risks on the design and manufacturing maturity prior to a production decision. These are manufacturability assessments and should be conducted in concert with the CDR and also later in EMD just prior to the Milestone C decision. Sources of data may include technical reviews and audits, Program Support Reviews, pre-award surveys, incremental PRRs, ICAs, trade-off studies, tooling plans, make-or-buy plans, manufacturing plans, and bills of material.\n     The results of the assessment of manufacturing readiness performed at the end of the TD Phase will be used as a baseline reference for this activity. The assessment should focus on program-wide manufacturing risks such as fabrication, assembly, integration and test operations; the supply chain performance; the maturity of manufacturing planning; the maturity of manufacturing management systems; adequacy of funding for manufacturing risk reduction efforts and other factors defined in MRL thread descriptions. Articles manufactured on a pilot line during EMD should be made using production materials, components, tooling, facilities and personnel."]
["$J$3",35]
["$K$3",36,"Low-Rate Initial Production (LRIP)","     At Milestone C the decision is made as to whether the program will proceed into the Production and Deployment Phase. The purpose of the Production and Deployment Phase is to achieve an operational capability that satisfies mission needs. A program may be structured with either one or two major decision points for this phase. The MDA 3-11\nfor Milestone C will decide if the program will enter LRIP or FRP. The target MRL for LRIP is 8 while the target is 9 for FRP.\n     If LRIP is required, to the extent practical, this production effort should be performed in a manner that uses designs, tooling, materials, components, facilities, and personnel that are representative of the FRP environment. The FRP decision requires that manufacturing risk is understood and that the manufacturing processes for the system be capable, in control, and affordable. Prior to the FRP decision, a manufacturing readiness assessment should be conducted to ensure any outstanding risks will not impact the programs ability to deliver FRP requirements.\n     Assessments of manufacturing readiness may be used to capture manufacturing product documentation. It is a best practice to incorporate the preservation of such manufacturing product technical data packages in the Data Management Strategy."]
["$L$3",37,"Full-Rate Production (FRP)","     The FRP decision requires that manufacturing risk is understood and that the manufacturing processes for the system be capable, in control, and affordable. Prior to the FRP decision, a manufacturing readiness assessment should be conducted to ensure any outstanding risks will not impact the programs ability to deliver FRP requirements.\n     Assessments of manufacturing readiness may be used to capture manufacturing product documentation. It is a best practice to incorporate the preservation of such manufacturing product technical data packages in the Data Management Strategy."]
["$A$4",38,"Technical Reviews"]
["$B$4",39]
["$C$4",40]
["$D$4",41]
["$E$4",42]
["$F$4",43,"ASR","Alternative Systems Review (ASR)\n     This review assesses preliminary materiel solutions that were identified during the MSA phase and assesses their potential for affordability, suitability, and operational effectiveness.  It also answers the question, “can this solution be developed in a timely manner at an acceptable level of risk?”  Again, the manufacturing feasibility is of critical importance in this review to increase the probability of meeting the affordability and schedule constraints of each alternative concept.\n\nSuccess criteria for the ASR prior to Milestone A are as follows:\n· Have the preliminary manufacturing processes and risks been identified for prototypes?\n· Have required investments for technology development, to mature design and manufacturing related technologies, been identified and funded?\n· Have initial producibility assessments of design concepts been completed?"]
["$G$4",44,"SRR/SFR","Systems Requirements Review (SRR)\n     The SRR is a multi-disciplined technical review to ensure that the system under review can proceed into initial systems development. The review considers whether all system requirements and performance requirements derived from the Initial Capabilities Document or draft Capability Development Document are defined and testable, and are consistent with cost, schedule, risk, technology readiness, and other system constraints. The manufacturing readiness is one of more important areas to address in determining whether the program can achieve its cost and schedule constraints. Analyzing whether manufacturing processes are mature enough is an essential element in determining whether a program will achieve its required performance within budget and schedule targets. A manufacturing readiness assessment results should be used at the SRR to update the program risk assessment, the Cost Analysis Requirements Description (CARD), and the program schedule. –Link to DAG\n\nSystems Functional Review (SFR)\n     The SFR is a multi-disciplined technical review to ensure that the system's functional baseline is established and has a reasonable expectation of satisfying the requirements of the Initial Capabilities Document or draft Capability Development Document within the allocated budget and schedule. Again, analyzing whether manufacturing processes are mature enough is an essential element in determining whether a program will achieve its required performance within budget and schedule targets. Manufacturing readiness assessment results should also be used at the SFR to update the program risk assessment for Engineering and Manufacturing Development (EMD), the Cost Analysis Requirements Description (CARD), and the program schedule."]
["$H$4",45,"PDR","Preliminary Design Review (PDR)\n     The PDR establishes the allocated baseline (hardware, software, human/support systems) and underlying architectures to ensure that the system under review has a reasonable expectation of satisfying the requirements within the currently allocated budget and schedule. The maturity of the manufacturing process and associated risk of attaining that maturity is an essential element in determining whether a program will achieve its required performance within budget and schedule targets.  Manufacturing readiness assessment results should also be used at the PDR to update the program risk assessment for Engineering and Manufacturing Development (EMD), the Cost Analysis Requirements Description (CARD), and the program schedule. \n\nAt the PDR prior to Milestone B the following questions apply:\n· Have the majority of manufacturing processes been defined and characterized?\n· Are initial manufacturing approaches documented?\n· Have producibility assessments of key technologies been completed?\n· Has a production cost model been constructed?\n· Can the industrial base support production of development articles?\n· Have long-lead and key supply chain elements been identified?"]
["$I$4",46,"CDR","Critical Design Review (CDR)\n     The CDR is to ensure that the system under review can proceed into system fabrication, demonstration, and test, and can meet the stated performance requirements within cost (program budget), schedule (program schedule), risk, and other system constraints. Results of the manufacturing assessments need to be used specifically to determine whether the system under review can meet cost, schedule, and performance objectives.  \n\nExit questions for the CDR prior to System Capability and Manufacturing Process Demonstration include:\n· Have the critical manufacturing processes that affect the key characteristics been identified and their capability to meet design tolerances determined?\n· Have process control plans been developed for critical manufacturing processes?\n· Have manufacturing processes been demonstrated in a production representative environment?\n· Are detailed trade studies and system producibility assessments underway?\n· Are materials and tooling available to meet pilot line schedule?\n· Has the system production cost model been updated, allocated to subsystem level, and tracked against targets?\n· Are long-lead procurement plans in place and has the supply chain been assessed?"]
["$J$4",47,"PRR/SVR","Production Readiness Review (PRR)\n     The PRR examines a program to determine if the design is ready for production and if the prime and major subcontractors have accomplished adequate production planning without incurring unacceptable risks that breach thresholds of schedule, performance, cost, or other established criteria. The review examines risk; it determines if production or production preparations identify unacceptable risks that might breach thresholds of schedule, performance, cost, or other established criteria. The review evaluates the full production-configured system to determine if it correctly and completely implements all system requirements. Since the PRR looks at all system requirements, it will look at more than manufacturing issues (e.g., software, sustainment, operational testing, etc.). The review determines whether traceability of final system requirements to the final production system is maintained.    \n\nThe following success criteria are associated with the PRR prior to Milestone C:\n· Is the detailed design producible within the production budget?\n· Are the production facilities ready and required workers trained?\n· Is detail design complete and stable enough to enter low rate production?\n· Is the supply chain established and stable with materials available to meet planned low rate production?\n· Have manufacturing processes been demonstrated and proven in a pilot line environment?\n· Have all producibility trade studies and risk assessments been completed?\n· Is the production cost model based upon the stable detailed design and been validated?\n\nSystem Verification Review (SVR)\n     The SVR is a multi-disciplined product and process assessment to ensure that the system under review can proceed into Low-Rate Initial Production and Full-Rate Production within cost (program budget), schedule (program schedule), risk, and other system constraints. \nGenerally this review is an audit trail from the Critical Design Review. It assesses the system final product, as evidenced in its production configuration, and determines if it meets the functional requirements (derived from the Capability Development Document and draft Capability Production Document) documented in the Functional, Allocated, and Product Baselines. \n\nTypical SVR success criteria include affirmative answers to the following exit questions: (1) Does the status of the technical effort and system indicate operational test success (operationally suitable and effective)? (2) Can the system, as it exists, satisfy the Capability Development Document/draft Capability Production Document? (3) Are adequate processes and metrics in place for the program to succeed? (4) Are the risks known and manageable? (5) Is the program schedule executable within the anticipated cost and technical risks? (6) Are the system requirements understood to the level appropriate for this review? (7) Is the program properly staffed? (8) Is the program's Non Recurring Engineering requirement executable with the existing budget? (9) Is the system producible within the production budget? ","https://acc.dau.mil/CommunityBrowser.aspx?id=638321"]
["$K$4",48,"PCA","Physical Configuration Audit (PCA)\n     The PCA is conducted around the time of the Full-Rate Production Decision. The PCA examines the actual configuration of an item being produced. It verifies that the related design documentation matches the item as specified in the contract. In addition to the standard practice of assuring product verification, the PCA confirms that the manufacturing processes, quality control system, measurement and test equipment, and training are adequately planned, tracked, and controlled. Results of the manufacturing assessments will be critical in performing this Audit. \n"]
["$L$4",49]
["$A$5",50,"Thread"]
["$B$5",51,"Sub-Thread"]
["$C$5",52,"MRL 1","MRL 1:  Basic Manufacturing Implications Identified \n\nThis is the lowest level of manufacturing readiness.  The focus is to address manufacturing shortfalls and opportunities needed to achieve program objectives.  Basic research (i.e., funded by budget activity) begins in the form of studies. \n\nBackground: \nThe system is very early in the Pre Material Solution Analysis (Pre-MSA) phase before the acquisition cycle begins, and it may not even be registered as a formal program. This phase of the Research and Development is usually associated with Basic Research.  Prior to MSA, manufacturing processes are limited to characterizing the state of the manufacturing risk of a potential product than assessing manufacturing aspects of the outcome of scientific discovery.  Alternatively, assessing manufacturing aspects of new technologies or materials may provide insight into new manufacturing processes that need to be developed to achieve innovative new products.   Activities in manufacturing design maturity are usually restricted to identifying research opportunities and making investment plans to support new approaches and technologies to support the manufacturing process."]
["$D$5",53,"MRL 2","MRL 2:  Manufacturing Concepts Identified \n\nThis level is characterized by describing the application of new manufacturing concepts.   Applied research translates basic research into solutions for broadly defined military needs.  Typically this level of readiness includes identification, paper studies and analysis of material and process approaches. An understanding of manufacturing feasibility and risk is emerging.\n\nBackground:\nThe system is early in the advanced research project, pre-Material Solution Analysis (Pre-MSA) phase before the acquisition cycle begins, and it may not even be registered as a formal program.  Prior to MSA, manufacturing processes are limited to characterizing the state of the manufacturing risk of a potential product than assessing manufacturing aspects of the outcome of scientific discovery.  Alternatively, assessing manufacturing aspects of new technologies or materials may provide insight into new manufacturing processes that need to be developed to achieve innovative new products.   This phase of the Research and Development is usually associated with Basic Research or Applied Research in Science and Technology. Manufacturing concepts will be validated through analytical paper studies."]
["$E$5",54,"MRL 3","MRL 3:  Manufacturing Proof of Concept Developed\n\nThis level begins the validation of the manufacturing concepts through analytical or laboratory experiments. This level of readiness is typical of technologies in Applied Research and Advanced Development. Materials and/or processes have been characterized for manufacturability and availability but further evaluation and demonstration is required.  Experimental hardware models have been developed in a laboratory environment that may possess limited functionality.  \n\nBackground:\nThe system is in the Pre-MSA phase before the acquisition cycle begins, and it still may not be registered as a formal program.  At this stage the program is primarily analytical, focusing on the proof of concept. Manufacturing concepts will be validated through analytical (paper studies) or early laboratory experiments.   Specific materials and/or processes are not defined and are very changeable.  At this stage, therefore, potential sources for a variety of materials and processes are being surveyed with no final decision made on any sources so an Analysis of Alternatives (AoA) will be conducted of potential solutions to address users’ needs. The task is to examine potential material solutions with the goal of identifying the most promising option that can best support program requirements.  The main purpose of MRLs in this phase is to assess the manufacturing feasibility of various alternatives being considered to understand the risks with proceeding with the selected solution(s).  The manufacturing risk(s) identified in this phase needs to be addressed in the Technology Development Phase risk mitigation efforts and reflected in all cost estimates."]
["$F$5",55,"MRL 4","MRL 4:  Capability to produce the technology in a laboratory environment\n\nThis level of readiness acts as an exit criterion for the Materiel Solution Analysis (MSA) Phase approaching a Milestone A decision.  Technologies should have matured to at least TRL 4.  This level indicates that the technologies are ready for the Technology Development Phase of acquisition.  At this point, required investments, such as manufacturing technology development, have been identified.  Processes to ensure manufacturability, producibility, and quality are in place and are sufficient to produce technology demonstrators.  Manufacturing risks have been identified for building prototypes and mitigation plans are in place. Target cost objectives have been established and manufacturing cost drivers have been identified.  Producibility assessments of design concepts have been completed.  Key design performance parameters have been identified as well as any special tooling, facilities, material handling and skills required.\n\nBackground:\nThe system is now in the Material Solution Analysis (MSA) phase of the acquisition cycle. The purpose of the MSA is to assess potential materiel solutions and to satisfy the phase-specific entrance criteria for the next program milestone designated by the Milestone Decision Authority. This phase refines the initial concept by beginning to conduct an Analysis of Alternatives (AoA) of potential solutions to address user’s needs. The AoA will examine potential material solutions to identify the most promising option to best support program requirements. Components or breadboards are being validated in a laboratory environment, and specific materials and/or processes are still in a state of flux; however, systems design personnel are homing in on a few preferred design approaches.  Potential sources for a materials and processes can be narrowed; however, options must be kept open until a final decision is made on the design. Special emphasis is also placed on the proposed material solution to analyze supply chain feasibility from a manufacturing perspective and determine manufacturing resources and technology needed. An Alternative Systems Review (ASR) may be performed to assess preliminary materiel solutions identified during the MSA regarding their potential for affordability, suitability, and operational effectiveness.  The ASR is an attempt to ensure the system requirements align with the customer’s needs and attempts to minimize the need to change requirements in the later acquisition phases. A strategy for Technology Development should be defined and it should address quality in a general way, probably from the general requirements perspective. The main purpose of MRLs in this phase is to assess the manufacturing feasibility of various alternatives being considered in order to understand the risks with proceeding with the selected solution(s).  Part of that feasibility assessment will identify workforce skills needed to produce demonstration articles in the Technology Development (TD) Phase and to forecast the needs for the production program.  The manufacturing risk identified in this phase needs to be addressed in the Technology Development Phase risk mitigation efforts and reflected in all cost estimates."]
["$G$5",56,"MRL 5","MRL 5:  Capability to produce prototype components in a production relevant environment\n\nThis level of maturity is typical of the mid-point in the Technology Development Phase of acquisition, or in the case of key technologies, near the mid-point of an Advanced Technology Demonstration (ATD) project.  Technologies should have matured to at least TRL 5.  The industrial base has been assessed to identify potential manufacturing sources.  A manufacturing strategy has been refined and integrated with the risk management plan. Identification of enabling/critical technologies and components is complete.  Prototype materials, tooling and test equipment, as well as personnel skills have been demonstrated on components in a production relevant environment, but many manufacturing processes and procedures are still in development.  Manufacturing technology development efforts have been initiated or are ongoing.  Producibility assessments of key technologies and components are ongoing.  A cost model has been constructed to assess projected manufacturing cost.\n\nBackground:\nThe system is now in the early stages of the Technology Development (TD) phase. The program has passed the MS A decision point.  The technology applied to product and manufacturing processes is under development.  The midpoint of an Advanced Technology Demonstration (ATD) may be reached if new technology is involved.  Potential sources of supply have been identified.  The purpose of this phase of the acquisition cycle is to assess, identify, and reduce technical risk from both a technology and manufacturing perspective, determine manufacturing capabilities to be integrated into a full system for both the prime and the supply chain, and demonstrate Critical Technologies (CTs) on prototypes.  This will be done by manufacturing and testing prototype units/components in a relevant environment.   Specific materials and/or processes may be in a state of flux; however, this is the stage where a formal Industrial Base Capabilities Assessment (IBCA) is done and an Analysis of Alternatives (AoA) has been performed to select the best design. At Milestone A, the Milestone Decision Authority (MDA) reviews the proposed materiel solution and the draft Technology Development Strategy (TDS) to determine if ready to proceed and has documented the decision in an Acquisition Decision Memorandum (ADM).  The strategy for Technology Development should address Quality including evaluation of customer quality requirements.  The TDS will undergo refinement during the TD phase.  The program will have undergone a Systems Requirements Review/System Functional Review (SRR/SFR) or is preparing for a PDR in the near future.  At this stage sources for a materials and processes are being fixed, but options may be kept open until the final design decision. Understanding the risk of achieving the production requirements and material maturity will be a key element of exiting this phase and assessments will be done to determine whether manufacturing technology development is required to achieve the user’s requirements.\n"]
["$H$5",57,"MRL 6","MRL 6:  Capability to produce a prototype system or subsystem in a production relevant environment\n\nThis MRL is associated with readiness for a Milestone B decision to initiate an acquisition program by entering into the Engineering and Manufacturing Development (EMD) Phase of acquisition.  Technologies should have matured to at least TRL 6.  It is normally seen as the level of manufacturing readiness that denotes acceptance into a preliminary system design.  An initial manufacturing approach has been developed. The majority of manufacturing processes have been defined and characterized, but there are still significant engineering and/or design changes in the system itself.  However, preliminary design of critical components has been completed and producibility assessments of key technologies are complete.  Prototype materials, tooling and test equipment, as well as personnel skills have been demonstrated on systems and/or subsystems in a production relevant environment.  A cost analysis has been performed to assess projected manufacturing cost versus target cost objectives and the program has in place appropriate risk reduction to achieve cost requirements or establish a new baseline. This analysis should include design trades.  Producibility considerations have shaped system development plans.  The Industrial Capabilities Assessment (ICA) for Milestone B has been completed. Long-lead and key supply chain elements have been identified.  \n\nBackground:\nThe system is nearing the end of the Technology Development (TD) phase and the system or subsystem model or prototype is now being demonstrated in a relevant environment.  Specific materials and/or processes are quantified, and the formal Industrial Base Capabilities Assessment (IBCA) for Milestone B is complete.  Industrial capability is largely in place to support development articles since sources for materials and processes are largely fixed and the program will have conducted a Preliminary Design Review (PDR).  Demonstrating critical manufacturing processes in a production relevant environment is critical to assessing manufacturing risk and giving confidence that program cost, schedule and performance requirements before proceeding into EMD.  This level of production realism is well beyond what is seen in a laboratory.  The emphasis is on addressing higher risk areas (e.g., more advanced technologies and newer manufacturing capabilities).  During this critical junction it is essential that the contractor(s) demonstrate the capability to build the product or a similar item (considering size, tolerances, quality levels, processes, and testing) in the facility that will be used during production.  At the end of the TD phase an assessment of manufacturing risk and readiness is conducted to baseline needed supply chain industrial capabilities and identify remaining required investments for every competing design or prototype that has conducted a PDR. Since the program is approaching MS B, it is important to make sure the Quality Management System is developing in parallel with the system design.  A Program Quality Plan should be developed and in place. Quality metrics have been identified, and improvement plans are under development.  Potential sources of supply have been evaluated for quality risk, and their ability to meet customer quality requirements.  Quality data on production of other similar products may provide insight into the supplier’s ability to meet our requirements.  Most importantly, at this stage of development, the quality thresholds should be established with reference to past performance or anticipated design tolerances and process yields. When assessing and reducing manufacturing risk with the options being considered, it is important understand future tooling, test equipment and facilities requirements. The program should exit the Technology Demonstration (TD) Phase when the technology and manufacturing processes for that program or increment have been assessed and demonstrated in a relevant environment and manufacturing risks have been identified. The program will be approaching a Critical Design Review in the near future.  "]
["$I$5",58,"MRL 7","MRL 7:  Capability to produce systems, subsystems, or components in a production representative environment\n\nThis level of manufacturing readiness is typical for the mid-point of the Engineering and Manufacturing Development (EMD) Phase leading to the Post-CDR Assessment. Technologies should be on a path to achieve TRL 7.  System detailed design activity is underway.  Material specifications have been approved and materials are available to meet the planned pilot line build schedule.  Manufacturing processes and procedures have been demonstrated in a production representative environment.  Detailed producibility trade studies and risk assessments are underway.  The cost model has been updated with detailed designs, rolled up to system level, and tracked against allocated targets. Unit cost reduction efforts have been prioritized and are underway. The supply chain and supplier quality assurance have been assessed and long-lead procurement plans are in place. Production tooling and test equipment design and development have been initiated.\n\nBackground:\nThe system is beginning Engineering and Manufacturing Development (EMD) and the system prototype is beginning demonstration in an operational environment.  EMD completes the development of a system, leveraging design considerations (quality, producibility and manufacturability), completes the full system integration, develops affordable and executable manufacturing (and quality) processes, completes system fabrication, assembly, test and evaluation.   Entrance into this phase depends on technology maturity, approved requirements, and full funding. Unless some other factor is overriding in its impact, the maturity of the technology determines the path to be followed.  MDA approval of the Acquisition Strategy is a requirement before final RFPs for EMD can be released.  Specific materials and/or processes are quantified, and the industrial base capability is being monitored.  A Critical Design Review (CDR) should have been conducted or is near to being conducted, and the design is rapidly maturing.  The manufacturing focus in this phase is to demonstrate that manufacturing processes will support program requirements and that manufacturing risk reduction initiatives are reflected in the acquisition strategy.  It is critical that at this stage of the acquisition cycle that the contractor maintains effective supply chain management processes and procedures.  Items are being designed and built in a production representative environment.  Manufacturing processes are maturing, and materials are being used in the fabrication and assembly operations.  The program has successfully achieved a MS B decision and the Quality Management System (QMS) is undergoing further development in parallel with the system design.  A Program Quality Plan and Quality Metrics have been developed. Quality metrics have been identified, and improvement plans identified prior to milestone B are being applied.  Quality targets are identified and progress on development units and other pre-production runs should show these targets are achievable.  Quality data on production of other similar products may provide insight into the prime’s and supplier’s abilities to meet quality targets.  Manufacturing exit criteria for this phase include a demonstration that processes are capable of producing affordable systems or components in a production representative environment and that they will be capable of producing in pilot line production.  Preparations are also beginning for the manufacture of qualification test hardware. \n"]
["$J$5",59,"MRL 8","MRL 8:  Pilot line capability demonstrated; Ready to begin Low Rate Initial Production\n\nThis level is associated with readiness for a Milestone C decision, and entry into Low Rate Initial Production (LRIP).  Technologies should have matured to at least TRL 7.  Detailed system design is essentially complete and sufficiently stable to enter low rate production.  All materials are available to meet the planned low rate production schedule.  Manufacturing and quality processes and procedures have been proven in a pilot line environment and are under control and ready for low rate production.  Known producibility risks pose no significant challenges for low rate production.  The engineering cost model is driven by detailed design and has been validated with actual data.  The Industrial Capabilities Assessment for Milestone C has been completed and shows that the supply chain is established and stable.  \n\nBackground:\nThe program is approaching a MS C decision and the industrial capabilities supporting the program should be defined and viable at this time.  The industrial base should be capable of beginning Low Rate Initial Production (LRIP).  Detailed design is complete for the majority of the system.  The design should be stable so that manufacturing equipment and technologies are mature at this time.  During this phase manufacturing assessments will demonstrate that the manufacturing processes will/or will not support production requirements and that processes are demonstrated in a pilot line. The use of MRLs will determine whether manufacturing processes are mature enough to achieve program cost, schedule, and performance objectives prior to entering LRIP. Some programs conduct a Production Readiness Review (PRR) as part of their systems engineering technical review at the end of EMD to determine if the program is ready for production. It is critical that at this point that the prime completes its assessment on lower tier supplier capability and capacity.  The contractor’s Quality Management System (QMS) is undergoing further development in parallel with the system design. A Program Quality Plan and Quality Metrics have been developed, and improvement plans identified prior to milestone B and CDR are being applied.  Quality targets are identified and data from production representative lines and pilot lines should show these targets are achievable.  Material planning systems to manage these materials should be ready to support the LRIP build. An MRL 8 evaluation should be used to support a Milestone C decision and the decision to begin LRIP.  Key manufacturing risks that could impact LRIP should be identified with mitigation plans in place.\n"]
["$K$5",60,"MRL 9","MRL 9:  Low rate production demonstrated; Capability in place to begin Full Rate Production\n\nAt this level, the system, component or item has been previously produced, is in production, or has successfully achieved low rate initial production.  Technologies should have matured to TRL 9.  This level of readiness is normally associated with readiness for entry into Full Rate Production (FRP).  All systems engineering/design requirements should have been met such that there are minimal system changes.  Major system design features are stable and have been proven in test and evaluation.  Materials are available to meet planned rate production schedules.  Manufacturing process capability in a low rate production environment is at an appropriate quality level to meet design key characteristic tolerances.  Production risk monitoring is ongoing.  LRIP cost targets have been met, and learning curves have been analyzed with actual data.  The cost model has been developed for FRP environment and reflects the impact of continuous improvement.\n\nBackground:\nThe program has successfully achieved a MS C decision, and if Milestone C approves Low Rate Initial Production (LRIP), a subsequent review and decision will authorize Full Rate Production (FRP).  The main manufacturing focus of this phase is to have an orderly increase in the production rate upon successfully completing operational testing.  Entrance into this phase depends on the following criteria: Acceptable performance in developmental test and evaluation and operational assessment; no significant manufacturing risks; manufacturing processes under control (if Milestone C is full-rate production) and production representative articles successfully demonstrated in LRIP; an approved Capability Production Document (CPD); acceptable interoperability; acceptable operational supportability; and demonstration that the system is affordable throughout the life cycle, fully funded, and properly phased for rapid acquisition. The CPD reflects the operational requirements, informed by EMD results, and details performance expected of the production system.  Manufacturing processes are mature, and mature materials are being used in the fabrication and assembly operations. The key is that design is stable and the system is affordable. The Quality Management System (QMS) is undergoing evaluation in parallel with the production system.  A Program Quality Plan and Quality Metrics have been developed. Quality metrics have been identified, and improvement plans identified prior to milestone C are being applied.  Quality targets are identified and data from production representative and pilot lines should show these targets are achievable entering LRIP.  The material selection and material resource planning system will also be successfully demonstrated on production representative articles in LRIP.\n"]
["$L$5",61,"MRL 10","MRL 10:  Full Rate Production demonstrated and lean production practices in place\n\nThis is the highest level of production readiness.  Technologies should have matured to TRL 9.  This level of manufacturing is normally associated with the Production or Sustainment phases of the acquisition life cycle.  Engineering/design changes are few and generally limited to quality and cost improvements.  System, components or items are in full rate production and meet all engineering, performance, quality and reliability requirements.  Manufacturing process capability is at the appropriate quality level.  All materials, tooling, inspection and test equipment, facilities and manpower are in place and have met full rate production requirements.  Rate production unit costs meet goals, and funding is sufficient for production at required rates.  Lean practices are well established and continuous process improvements are ongoing.\n\nBackground:\nAt Milestone C the decision is made as to whether the program will proceed into the Production and Deployment Phase.  The purpose of the Production and Deployment Phase is to achieve an operational capability that satisfies mission needs.  Technologies have matured to TRL 9, and hardware qualification and Initial Operational Testing should have been completed.  At this level, the system, component or item has been in successful Full-Rate Production (FRP), at a rate adequate meet users’ deployment schedules. The main manufacturing focus of this phase is to provide the User quality products on cost and schedule. Manufacturing processes are mature, and the focus is now on continuous improvement using Lean/Six Sigma and other techniques.  The Quality Management System (QMS) is capably ensuring quality products are delivered and identifying opportunities for continued improvements in production processes.  A Program Quality Plan is maintained and Quality Metrics are reviewed, and demonstrate control of the manufacturing process.  Collection of statistical process control data, and demonstrated control and capability of other critical processes is ongoing.  Achieving MRL 10 (the manufacturing Promised Land) means there are no significant manufacturing risks at this stage of the system life cycle.\n"]
["$A$6",62]
["$B$6",63,"Technology\nMaturity "]
["$C$6",64,"Should be assessed at TRL 1.","TRL 1:  Basic principles observed and reported\n\nDescription:\nLowest level of technology readiness. Scientific research begins to be translated into applied research and development (R&D). Examples might include paper studies of a technology’s basic properties.\n\nSupporting Information:\nPublished research that identifies the principles that underlie this technology. References to who, where, when.\n\nExamples of ‘Objective Evidence’:\n•  TRA Report is TRL of 1 or greater\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Is the Technology Readiness at TRL 1 or greater?\n"]
["$D$6",65,"Should be assessed at TRL 2.","TRL 2:  Technology concept or application formulated\n\nDescription:\nInvention begins. Once basic principles are observed, practical applications can be invented. Applications are speculative, and there may be no proof or detailed analysis to support the assumptions. Examples are limited to analytic studies.\n\nSupporting Information\nPublications or other references that outline the application being considered and that provide analysis to support the concept.\n\nExamples of ‘Objective Evidence’:\n•  TRA Report is TRL of 2 or greater\n\nQuestions (MRL Criteria in the form of a question):\n\n1. Is the Technology Readiness at TRL 2 or greater?\n"]
["$E$6",66,"Should be assessed at TRL 3.","TRL 3:  Analytical and experimental critical function and/or characteristic proof of concept\n\nDescription:\nActive R&D is initiated. This includes analytical studies and laboratory studies to physically validate the analytical predictions of separate elements of the technology. Examples include components that are not yet integrated or representative.\n\nSupporting Information:\nResults of laboratory tests performed to measure parameters of interest and comparison to analytical predictions for critical subsystems. References to who, where, and when these tests and comparisons were performed.\n\nExamples of ‘Objective Evidence’:\n•  TRA Report is TRL of 3 or greater\n\nQuestions (MRL Criteria in the form of a question):\n1. Is the Technology Readiness at TRL 3 or greater?\n"]
["$F$6",67,"Should be assessed at TRL 4.","TRL 4:  Component or breadboard validation in a laboratory environment\n\nDescription:\nBasic technological components are integrated to establish that they will work together. This is relatively “low fidelity” compared with the eventual system. Examples include integration of “ad hoc” hardware in the laboratory.\n\nSupporting Information:\nSystem concepts that have been considered and results from testing laboratory-scale breadboard(s). References to who did this work and when. Provide an estimate of how breadboard hardware and test results differ from the expected system goals.\n\nExamples of ‘Objective Evidence’:\n•TRA Report is TRL of 4 or greater\n\nQuestions (MRL Criteria in the form of a question):\n\n1. Is the Technology Readiness at TRL 4 or greater?\n\n"]
["$G$6",68,"Should be assessed at TRL 5.","TRL 5:  Component or breadboard validation in a relevant environment\n\nDescription:\nFidelity of breadboard technology increases significantly. The basic technological components are integrated with reasonably realistic supporting elements so they can be tested in a simulated environment. Examples include “high-fidelity” laboratory integration of components.\n\n\nSupporting Information:\nResults from testing laboratory breadboard system are integrated with other supporting elements in a simulated operational environment. How does the “relevant environment” differ from the expected operational environment? How do the test results compare with expectations? What problems, if any, were encountered? Was the breadboard system refined to more nearly match the expected system goals?\n\nExamples of ‘Objective Evidence’:\n•TRA Report is TRL of 5 or greater\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Is the Technology Readiness at TRL 5 or greater?\n\n\n"]
["$H$6",69,"Should be assessed at TRL 6.","TRL 6:  System or subsystem model or prototype demonstrated in a relevant environment\n\nDescription:\nRepresentative model or prototype system, which is well beyond that of TRL 5, is tested in a relevant environment. Represents a major step up in a technology’s demonstrated readiness. Examples include testing a prototype in a high-fidelity laboratory environment or in a simulated operational environment.\n\nSupporting Information:\nResults from laboratory testing of a prototype system that is near the desired configuration in terms of performance, weight, and volume. How did the test environment differ from the operational environment? Who performed the tests? How did the test compare with expectations? What problems, if any, were encountered? What are/were the plans, options, or actions to resolve problems before moving to the next level?"]
["$I$6",70,"Should be assessed at TRL 7","TRL 7:  System prototype demonstration in an operational environment\n\n\nDescription:\nPrototype near or at planned operational system. Represents a major step up from TRL 6 by requiring demonstration of an actual system prototype in an operational environment (e.g., in an air-craft, in a vehicle, or in space).\n\n\nSupporting Information:\nResults from testing a prototype system in an operational environment. Who performed the tests? How did the test compare with expectations? What problems, if any, were encountered? What are/were the plans, options, or actions to resolve problems before moving to the next level?\n\n\nExamples of ‘Objective Evidence’:\n•TRA Report is TRL of 7 or greater\n"]
["$J$6",71,"Should be assessed at TRL 7 or 8.","TRL 7:  System prototype demonstration in an operational environment\n\nDescription:\nPrototype near or at planned operational system. Represents a major step up from TRL 6 by requiring demonstration of an actual system prototype in an operational environment (e.g., in an air-craft, in a vehicle, or in space).\nSupporting Information:\nResults from testing a prototype system in an operational environment. Who performed the tests? How did the test compare with expectations? What problems, if any, were encountered? What are/were the plans, options, or actions to resolve problems before moving to the next level?\nExamples of ‘Objective Evidence’:\n•\tTRA Report is TRL of 7 or greater\n"]
["$K$6",72,"Should be assessed at TRL 8 or 9.","TRL 8.  Actual system completed and qualified through test and demonstration.\n\n\nDescription:\nTechnology has been proven to work in its final form and under expected conditions. In almost all cases, this TRL represents the end of true system development. Examples include developmental test and evaluation (DT&E) of the system in its intended weapon system to determine if it meets design specifications.\n\n\nSupporting Information:\nResults of testing the system in its final configuration under the expected range of environmental conditions in which it will be expected to operate. Assessment of whether it will meet its operational requirements. What problems, if any, were encountered? What are/were the plans, options, or actions to resolve problems before finalizing the design?\n\n\nExamples of ‘Objective Evidence’:\n•TRA Report is TRL of 8 or greater\n\n\n"]
["$L$6",73,"Should be assessed at TRL 9.","TRL 9:  Actual system “flight proven” through successful mission operations\n\nDescription:\nActual application of the technology in its final form and under mission conditions, such as those encountered in operational test and evaluation (OT&E). Examples include using the system under operational mission conditions.\n\nSupporting Information:\nOT&E reports.\n\nExamples of ‘Objective Evidence’:\n•TRA Report is TRL of 9\n"]
["$A$7",74,"A- Technology and Industrial Base","Technology and the Industrial Base: Requires an analysis of the capability of the national technology and industrial base to support the design, development, production, operation, uninterrupted maintenance support of the system and eventual disposal (environmental impacts).\n\nTechnology and the Industrial Base includes industrial base issues and manufacturing technology development."]
["$B$7",75,"A.1 - Industrial base"]
["$C$7",76,"Global trends in emerging industrial base capabilities identified.","Purpose: \nTo begin to research manufacturing concepts to use in exploring potential solutions. The focus is to address manufacturing shortfalls and opportunities needed to achieve program objectives.  Basic research (i.e., funded by budget activity) begins in the form of studies.  This information will give decision makers insight into the risk and requirements of manufacturing concepts and will allow them to explore potential solutions for concepts early in the S&T phase. \n\n\nSources of Information:  \nThe S&T community, both S&T sponsors like the Service S&T organizations and research organizations in academia and industry provide a plethora of collaborative technical information in the state of the art in manufacturing science and technology.  Basic research (i.e., funded by budget activity) begins in the form of studies. \n\n\nExamples of ‘Objective Evidence’:\n•  Global trends in emerging industrial base capabilities identified and documented\n\n\nQuestions (MRL Criteria in the form of a question): \n1. Have global trends in emerging industrial base capabilities been identified?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$7",77,"Potential industrial base capability gaps identified.","Purpose: \nTo begin to identify new manufacturing requirements (e.g. concepts) to build future products and to use this information to explore potential solutions. This information will give decision makers insight into the risk and requirements of new manufacturing concepts and will allow them to explore potential solutions for concepts early in the S&T phase. The S&T community needs to have a basic understanding on the manufacturing requirements and the risk and implications to proceed with the proposed approach.  \n\nSources of Information:  \nThe S&T community, both S&T sponsors like the Service S&T organizations and research organizations in academia and industry provide a plethora of collaborative technical information in the state of the art in manufacturing science and technology.  Professional organizations provide the ideal forum for exchange of new ideas and approaches to exploring the boundaries of discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further the state of knowledge. \n\nExamples of ‘Objective Evidence’:\n•  Potential industrial base capability gaps identified and documented\n\nQuestions (MRL Criteria in the form of a question): \n1. Have potential industrial base capability gaps been identified?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$E$7",78,"Industrial base capabilities for potential sources identified for system concepts. ","Purpose:  \nTo identify possible sources for a variety of materials that may be used in the system and understand the state of the art with respect to processes.  Understanding the materials and processes (M&P) state of the art provides a foundation of essential information for an Analysis of Alternatives with respect to system design and capabilities.\n\nSources of Information:  \nThe majority of the information to an industrial base assessment at this early stage will come from the contractor’s design engineers and vendor/supply managers.  \n\nExamples of ‘Objective Evidence’:\n•  Industrial base capabilities for potential sources for system concepts documented\n \nQuestions (MRL Criteria in the form of a question): \n1. Have industrial base capabilities for potential sources been identified for system concepts?\n\nAdditional Considerations:\n• Do you possess the most recent information regarding industrial base capabilities?\n• Are any strategic or critical materials obtained only from sole or foreign sources?\n• Are sources for critical materials financially sound with multiple products and limited dependence on military products for profitability?\n• Do critical materials have significant commercial application or have application to more than one service?\n• Are critical materials on the list for diminishing manufacturing sources/material shortages?\n• What are the limitations on using certain materials for US defense purposes?\n\nLessons Learned: \n• Sources previously available for certain materials and/or processes may no longer exist; therefore, it is essential that the industrial base analysis be up-to-date.\n• For certain systems/programs, the use of foreign sources may either be proscribed or limited due to the strategic nature of the materials. \n• Foreign sources for some materials or processes may refuse to sell to US weapons programs, in which case, workarounds must be established early in the system life-cycle."]
["$F$7",79,"Industrial base capabilities surveyed for preferred materiel solution, key technologies, components, and/or key processes. Industrial base capability risks and issues included in AoA.","Purpose:  \nTo narrow down possible sources for materials and processes applying to Critical Technologies (CTs) (a.k.a, CTEs), from the TRA, for preferred system design alternatives. Understanding the materials and processes (M&P) availability of sources provides essential information on key processes and technology gaps that may exist when one does an Analysis of Alternatives (AoA) in the Material Solution Analysis (MSA) phase. The Alternative Systems Review (ASR) also answers the question, “can this solution be developed in a timely manner at an acceptable level of risk?”  Manufacturing feasibility is of critical importance in this review to increase the probability of meeting the cost and schedule constraints for each alternative concept.\n\n\nSources of Information:\n The majority of the information to an industrial base assessment at this program stage will come from the contractor’s design engineers and suppliers/supply managers. Additional resources include the DoD Manufacturing and Industrial Base Policy (MIBP) office, the Defense Contract Management Agency (DCMA), industrial base analysts and planners within the military departments and other DOD components such as the Missile Defense Agency and Defense Logistics Agency. \n\n\nExamples of ‘Objective Evidence’:\n• Survey documentation for industrial base capabilities.\n• Gaps, risks, and issues identified for key technologies, components, and/or key processes documented\n•  Analysis of Alternatives (AoA)\n\n\nQuestions (MRL Criteria in the form of a question): \n1. Have industrial base capabilities been surveyed for preferred materiel solution, key technologies, components, and/or key processes?\n2. Are industrial base capability risks and issues included in AoA?\n\n\nAdditional Considerations:\n• Are industrial base studies current and do you possess the most recent information regarding industrial capabilities that apply to CTs?\n• Are any strategic/critical materials needed for CTs obtained only from sole/foreign sources?\n• Are sources for critical materials financially sound with multiple products and limited dependence on military products for profitability?\n• Do critical materials have significant commercial application or have application to more than one service?\n• Are critical materials on the list for diminishing manufacturing sources/material shortages?\n• What are the limitations on using certain materials for US defense purposes?\n• Have the preliminary manufacturing processes and risks been identified for prototypes?\n• Have required investments for technology development, to mature design and manufacturing related technologies, been identified and funded?\n• Have initial producibility assessments of design concepts been completed?\n\n\nLessons Learned: \n• Sources previously available for certain materials and/or processes may no longer exist; therefore, it is essential that the industrial base analysis be up-to-date.\n• For certain systems/programs, the use of foreign sources may either be proscribed or limited due to the strategic nature of the materials. \n• Foreign sources for some materials or processes may refuse to sell to US weapons programs, in which case, workarounds must be established early in the system life-cycle.\n• As designs fluctuate, Industrial Capability Analyses need to be constantly updated.\n• Fragility of the IB and criticality of critical components needs to be considered in the industrial base capabilities assessment.\n"]
["$G$7",80,"Industrial base capabilities assessment initiated to identify potential manufacturing sources. Sole/single/ foreign source vendors and vendors of technologies with potential obsolescence issues identified and planning initiated to minimize risks.  ","Purpose:  \nTo perform a thorough assessment of sources for materials and processes (M&P) applying to Critical Technologies (CTs) (a.k.a, CTEs), from the TRA, of the major system design alternatives. Understanding capability of the defense industrial base to support production of various alternative designs is crucial information for the acquisition decision makers. For applicable products, the Acquisition Strategy should also address the approach to making production rate and quantity changes (i.e., surge) in response to contingency needs.\n\n\nSources of Information:\nThe majority of the information acquired in support of the industrial base capabilities assessment comes from the contractor’s manufacturing/industrial engineering, quality, and subcontractor/ supply chain management departments.  Additional resources include the DoD Manufacturing and Industrial Base Policy (MIBP) office, the Defense Contract Management Agency (DCMA), industrial base analysts and planners within the military departments and other DOD components such as the Missile Defense Agency and Defense Logistics Agency.\n\n\nExamples of ‘Objective Evidence’:\n•  Industrial base capabilities assessment\n•  Documentation for sole/single/foreign sources with potential obsolescence issues\n•  Planning documentation to minimize the risks associated with sole/single/foreign sources\n•  Industrial base capabilities assessment with Program Memo\n\n\nQuestions (MRL Criteria in the form of a question): \n1. Has an assessment of industrial base capabilities been initiated to identify potential manufacturing sources?\n2. Have sole/single/ foreign source vendors and vendors of technologies with potential obsolescence issues been identified?\n3. Does sole/single/ foreign source vendors and vendors of technologies with potential obsolescence issues have planning initiated to minimize risks?\n\n\nAdditional Considerations:\n•  Are industrial base studies current and do you possess the most recent information regarding industrial capabilities that apply to CTs?\n•  Are any strategic or critical materials needed for CTs obtained only from sole/foreign sources?\n•  Is there likely to be a future requirement to surge production of this system?\n•  What is the viability of any associated essential industrial/technological capabilities?\n•  What is the potential viability of non-selected firms as enduring competitors for defense products?\n•  What span of time between current and potential future contract awards should be considered that will make selection critical to supplier business decisions?\n•  Are there other businesses of the same type or emerging capabilities that could serve as a replacement solution?\n•  What decisions will impact a supplier's future viability (jeopardize future competitiveness or do not provide a sufficient business case to keep the capabilities/unit around for the future)?\n•  What decisions will establish new industrial capabilities (develop new facilities, demonstrate and \"productionize\" new technologies, preserve health of the industrial base)?\n•  What priority under the Defense Priorities and Allocations System (DPAS) is this program likely to have (e.g., DO, DX, etc.)?\n\n\nLessons Learned: \n•  Sources previously available for certain materials and/or processes may no longer exist; therefore, it is essential that the industrial base analysis be up-to-date.\n•  For certain systems/programs, the use of foreign sources may either be proscribed or limited due to the strategic nature of the materials. \n•  Foreign sources for some materials or processes may refuse to sell to US weapons programs, in which case, workarounds must be established early in the system life-cycle.\n•  As designs fluctuate, Industrial Base Capability Analyses need to be constantly updated.\n•  Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n•  Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished industrial base capabilities assessment.\n•  Fragility of the IB and criticality of critical components need to be considered in the industrial base capabilities assessment.\n"]
["$H$7",81,"Industrial base capabilities assessment for MS B completed. Industrial capability in place to support manufacturing of development articles. Plans to minimize sole/single/foreign sources and obsolescence issues complete. Need for sole/single/foreign sources justified.  Potential alternative sources identified.  ","Purpose:  \nAssure a thorough assessment of sources for materials and processes that apply to Critical Technologies (CTs) (a.k.a, CTEs), from the TRA, has been completed and that viable sources and potential alternatives have been identified. Assuring that the capability of the defense industrial base to support production of system and subsystem designs is crucial for Milestone B decision makers and should be an exit criterion for PDR.\n\n\nSources of Information: \nThe majority of the information acquired in support of the industrial base capabilities assessment comes from the contractor’s manufacturing/industrial engineering, quality, and subcontractor/ supply chain management departments.  Additional resources include the DoD Manufacturing and Industrial Base Policy (MIBP) office, the Defense Contract Management Agency (DCMA), industrial base analysts and planners within the military departments and other DOD components such as the Missile Defense Agency and Defense Logistics Agency.\n\n\nExamples of ‘Objective Evidence’:\n•  Industrial base capabilities assessment\n•  Supplier management policy/procedure\n•  Industrial base capabilities assessment with Program Memo\n•  TRA Report is TRL of 6 or greater\n\n\nQuestions (MRL Criteria in the form of a question): \n1.  Is the assessment of industrial base capabilities to support the Milestone (MS) B decision complete?\n2.  Is there industrial capability in place that can support the manufacturing of (technology) development articles?\n3.  Are the plans to minimize sole, single, and/or foreign sources complete?\n4.  Are the plans to minimize potential obsolescence issues complete? \n5.  Has the need for sole, single, and/or foreign sources been justified?\n6.  Have potential alternative sources been identified?\n\n\nAdditional Considerations:\n•  Have the majority of manufacturing processes been defined and characterized?\n•  Are initial manufacturing approaches documented?\n•  Have producibility assessments of key technologies been completed?\n•  Has a production cost model been constructed?\n•  Can the industrial base support production of development articles?\n•  Have long-lead and key supply chain elements been identified? \n•  What decisions will impact a supplier's future viability (jeopardize future competitiveness or do not provide a sufficient business case to keep the capabilities/unit around for the future)?\n•  What is the certainty of future demand for key products given an economic turndown?\n•  What decisions will establish new industrial capabilities (develop new facilities, demonstrate and \"productionize\" new technologies, preserve health of the industrial base)?\n\n\nLessons Learned: \n•  As designs change, industrial base capabilities assessments need to be constantly updated.\n•  Failure to consider Industrial base capability as part of the PDR may lead to jeopardizing future production schedules.\n•  Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n•  Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished industrial base capabilities assessment.\n•  Fragility of the IB and criticality of critical components need to be considered in the industrial base capabilities assessment.\n"]
["$I$7",82,"Industrial capability to support production analyzed. Sole/single/foreign sources, source stability, and obsolescence issues are assessed/monitored. Potential alternate sources developed if necessary. ","Purpose:  \nAssure the industrial base is stable and viable sources and that potential alternatives have been identified.  Assuring that the defense industrial base is stable to support production of system designs should be an exit criterion for CDR.\n\n\nSources of Information: \nThe majority of the information acquired in support of the industrial base capabilities assessment comes from the contractor’s manufacturing/industrial engineering, quality, and subcontractor/ supply chain management departments.  Additional resources include the DoD Manufacturing and Industrial Base Policy (MIBP) office, the Defense Contract Management Agency (DCMA), industrial base analysts and planners within the military departments and other DOD components such as the Missile Defense Agency and Defense Logistics Agency.\n\n\nExamples of ‘Objective Evidence’:\n•Industrial base capabilities assessment to support production\n•Supplier management policy/procedure\n•Industrial base capabilities assessment with Program Memo\n\n\nQuestions (MRL Criteria in the form of a question): \n1. Has the industrial capability needed to support production been analyzed?\n2. Have all sole, single, and foreign sources been assessed?\n3. Is the stability of all sole, single, and foreign sources being monitored?\n4. If analyses of industrial capability, sources, and obsolescence indicate risks or issues, have potential alternate sources been developed?\n\n\n\nAdditional Considerations:\n• Have the critical manufacturing processes that affect the key characteristics been identified and their capability to meet design tolerances determined?\n• Are materials and tooling available to meet pilot line schedule?\n• Has the system production cost model been updated, allocated to subsystem level, and tracked against targets?\n• Are long-lead procurement plans in place and has the supply chain been assessed?\n\n\nLessons Learned: \n• As designs change, industrial base capabilities assessments need to be constantly updated.\n• Failure to consider Industrial base capability as part of the CDR may lead to jeopardizing future production schedules.\n• Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n• Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished industrial base capabilities assessment.\n• Fragility of the IB and criticality of critical components need to be considered in the industrial base capabilities assessment.\n"]
["$J$7",83,"Industrial base capability assessment for MS C completed. Industrial capability is in place to support LRIP.  Sources are available, including multi-sourcing where cost-effective or necessary to mitigate risk.","Purpose:  \nTo demonstrate that the industrial base capabilities required to support the program have been analyzed and assessed to be adequate to achieve LRIP objectives. Approaching the end of EMD, IB considerations are now focused on the prime integrating contractor and supply chain team for production, with consideration of alternate sources required to mitigate specific risks.  A “deep dive” analysis can provide the Program Manager with information on issues at the lower tiers of the industrial base that might not otherwise be discovered.  This information is necessary to determine whether the program has a stable subcontractor base and assured availability of materials for all critical items.\n\n\nSources of Information: \nThe majority of the information acquired in support of the industrial base capabilities assessment comes from the contractor’s manufacturing/industrial engineering, quality, and subcontractor/ supply chain management departments.  Additional resources include the DoD Manufacturing and Industrial Base Policy (MIBP) office, the Defense Contract Management Agency (DCMA), industrial base analysts and planners within the military departments and other DOD components such as the Missile Defense Agency and Defense Logistics Agency.\n\n\nExamples of ‘Objective Evidence’:\n•Industrial base capabilities assessment for Milestone C completed\n•Industrial capability in place to support Low Rate Initial Production (LRIP)\n•All necessary sources available, including multi-sourcing where cost-effective or necessary, to mitigate risk \n•Supplier management policy/procedure\n\n\nQuestions (MRL Criteria in the form of a question): \n1. Is an assessment of industrial base capabilities to support the Milestone C decision complete?\n2. Does analysis show that industrial capabilities are in place to support Low Rate (Initial) Production?\n3. Are sources available?\n4. When cost-effective or needed to mitigate risks, are multiple sources included?\n\n\n\nAdditional Considerations:\n•Does the industrial base capabilities assessment include an analysis of anticipated Subcontractors, Suppliers and Vendors?\n•Have all subcontractors/suppliers been identified? Are additional suppliers available for all critical items?\n•Have all Sole/Single/Foreign Sources been identified and have alternative plans been made for parts available from foreign sources?  Are these sources being monitored for stability?\n•Are industrial capabilities and capacity adequate to support Program needs?\n•Have contingency program requirements been validated and documented?\n•Have Long-Lead items been analyzed?\n•Are plans in effect to mitigate potential program schedule impacts?\n•Has multi-sourcing been employed where possible?\n•Have delivery schedules been demonstrated to support the LRIP build?\n•Have impacts of this acquisition program on the industrial base been analyzed?\n•Does the industrial base capabilities assessment address industrial Sector, and Individual Enterprise trends?\n•Does the industrial base capabilities assessment address environmental impacts?\n•Does the industrial base capabilities assessment address potential program stretch-out and re-start challenges?\n•Has the competition plan been updated?\n•Have co-development/co-production opportunities been updated?\n•Are there any proprietary processes that could impact government data rights?\n\n\nLessons Learned: \n•One of the important steps for achieving this requirement is to work with the contractor/ supplier to define the supply chain/industrial base for the program, with particular emphasis on critical/pacing suppliers and long-lead items.  \n•Earlier industrial base capabilities assessments for the program should have identified broader industrial base issues, such as sector trends and alternate sources of parts and material.  At this point in the program, the industrial base capabilities assessment should focus on the financial viability of the contractor team/supply chain in-place going into production.\n•At the higher levels of the industrial chain, key issues include knowledge of technology/ product refresh cycles and supplier turnover (DMSMS/obsolescence issues).  \n•At the lower levels, where the industrial base often necks down to a single source, the industrial base capabilities assessment is often the best way to identify potential risks (such as changes environmental laws impacting the product/process) before they become problems.\n•Fragility of the IB and criticality of critical components need to be considered in the industrial base capabilities assessment.\n"]
["$K$7",84,"Industrial capability assessment for FRP has been completed and capability is in place to support start of FRP. ","Purpose:  \nThe intent is to demonstrate that the industrial base capabilities required to support the program have been analyzed and assessed to be adequate to achieve FRP objectives. After the MS C decision, industrial base considerations are now focused on the prime integrating contractor and supply chain team for full rate production, with consideration of alternate sources required to mitigate specific risks.     \n\n\nSources of Information: \nThe majority of the information acquired in support of the industrial base capabilities assessment comes from the contractor’s manufacturing/industrial engineering, quality, and subcontractor/ supply chain management departments.  Additional resources include the DoD Manufacturing and Industrial Base Policy (MIBP) office, the Defense Contract Management Agency (DCMA), industrial base analysts and planners within the military departments and other DOD components such as the Missile Defense Agency and Defense Logistics Agency.\n\n\nExamples of ‘Objective Evidence’:\n•Industrial base capabilities assessments\n•Supplier management policy/procedure\n•Industrial base capabilities assessment with Program Memo\n\n\nQuestions:\n1. Is an assessment of industrial base capabilities to support the Full Rate Production (FRP) decision complete?\n2. Is the industrial capability in place to support the start of Full Rate Production (FRP)?\n\n\n\nAdditional Considerations: \n•Have all subcontractors/suppliers been identified? Are back-ups available for all critical items?\n•Are Sole/Single/Foreign Sources being monitored for stability?\n•Are industrial capabilities and capacity adequate to support program needs?\n•Have contingency program requirements been validated and documented?\n•Are plans in effect to mitigate potential program schedule impacts?\n•Has multi-sourcing been employed where possible?\n•Have delivery schedules been demonstrated to support the FRP build?\n•Have impacts of this acquisition program on the industrial base been analyzed?\n•Does the industrial base capabilities assessment address environmental impacts?\n•Does the industrial base capabilities assessment address potential program stretch-out and re-start challenges?\n\n\nLessons Learned: \n•One of the important steps for achieving this requirement is to work with the contractor/ supplier to define the supply chain/industrial base for the program, with particular emphasis on critical/pacing suppliers and long-lead items.  \n•Earlier industrial base capabilities assessments for the program should have identified broader industrial base issues, such as sector trends and alternate sources of parts and material.  In this regard, at this point in the program, the industrial base capabilities assessment should focus on the financial viability of the contractor team/supply chain in-place going into production.\n•At the higher levels of the industrial chain, key issues include knowledge of technology/ product refresh cycles and supplier turnover (DMSMS/obsolescence issues).  \n•At the lower levels, where the industrial base often necks down to a single source, the industrial base capabilities assessment is often the best way to identify potential risks (such as changes environmental laws impacting the product/process) before they become problems.\n•Fragility of the IB and criticality of critical components need to be considered in the industrial base capabilities assessment.\n"]
["$L$7",85,"Industrial capability supports FRP and is assessed to support modifications, upgrades, surge and other potential manufacturing requirements. ","Purpose:  \nThe intent is to demonstrate that the industrial base capabilities required to support the program have been analyzed and assessed to be adequate to achieve FRP objectives. After the MS C decision, industrial base considerations are now focused on the prime integrating contractor and supply chain team for full rate production, with consideration of alternate sources required to mitigate specific risks. \n\n\nSources of Information: \nThe majority of the information acquired in support of the industrial base capabilities assessment comes from the contractor’s manufacturing/industrial engineering, quality, and subcontractor/ supply chain management departments.  Additional resources include the DoD Manufacturing and Industrial Base Policy (MIBP) office, the Defense Contract Management Agency (DCMA), industrial base analysts and planners within the military departments and other DOD components such as the Missile Defense Agency and Defense Logistics Agency.\n\n\nExamples of ‘Objective Evidence’:\n•Industrial base capabilities assessment\n•Supplier management documentation\n•Industrial base capabilities assessment to support modifications, upgrades, surge, and other potential manufacturing requirements\n\n\nQuestions (MRL Criteria in the form of a question): \n1. Does analysis show that industrial capabilities are supporting Full Rate Production (FRP)?\n2. Has the industrial capability been assessed for capability to support modifications, upgrades, surge quantities, and other potential manufacturing requirements?\n\n\nAdditional Considerations: \n• Have all subcontractors/suppliers been identified? Are back-ups available for all critical items?\n• Are Sole/Single/Foreign Sources being monitored for stability?\n• Are industrial capabilities and capacity adequate to support program needs?\n• Have contingency program requirements been validated and documented?\n• Are plans in effect to mitigate potential program schedule impacts?\n• Have delivery schedules been demonstrated to support the FRP build?\n• Have impacts of this acquisition program on the industrial base been analyzed?\n\nLessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier to define the supply chain/industrial base for the program, with particular emphasis on critical/pacing suppliers and long-lead items.  \n•Earlier industrial base capabilities assessments for the program should have identified broader industrial base issues, such as sector trends and alternate sources of parts and material.  In this regard, at this point in the program, the industrial base capabilities assessment should focus on the financial viability of the contractor team/supply chain in-place going into production.\n•At the higher levels of the industrial chain, key issues include knowledge of technology/ product refresh cycles and supplier turnover (DMSMS/obsolescence issues).  \n•At the lower levels, where the industrial base often necks down to a single source, the industrial base capabilities assessment is often the best way to identify potential risks (such as changes environmental laws impacting the product/process) before they become problems.\n•Fragility of the industrial base and criticality of critical components need to be considered in the industrial base capabilities assessment.\n"]
["$A$8",86]
["$B$8",87,"A.2 - Manufacturing Technology Development"]
["$C$8",88,"Global trends in manufacturing science and technology identified (i.e., concepts, capabilities).","Purpose: \nTo begin to research manufacturing concepts to use in exploring potential solutions. The focus is to address manufacturing shortfalls and opportunities needed to achieve program objectives.  Basic research (i.e., funded by budget activity) begins in the form of studies.  This information will allow the development of needed manufacturing technology in the S&T phase. \n\n\nSources of Information:  \nThe S&T community, both S&T sponsors like the Service S&T organizations and research organizations in academia and industry provide a plethora of collaborative technical information in the state of the art in manufacturing technology development.  Basic research (i.e., funded by budget activity) begins in the form of studies. \n\n\nExamples of ‘Objective Evidence’:\n•  Global trends in manufacturing science and technology been identified (i.e., concepts, capabilities) identified and documented \n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have global trends in manufacturing science and technology been identified (i.e., concepts, capabilities)?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$8",89,"Potential manufacturing science and technology gaps identified.","Purpose: \nTo begin identify new manufacturing requirements (e.g. concepts) to build future products and to use this information to explore potential solutions. This information will give decision makers insight into the risk and requirements of new manufacturing concepts and will allow them to explore potential solutions for concepts early in the S&T phase. The S&T community needs to have a basic understanding on the manufacturing requirements and the risk and implications to proceed with the proposed approach.  \n\nSources of Information:  \nThe S&T community, both S&T sponsors like the Service S&T organizations and research organizations in academia and industry provide a plethora of collaborative technical information in the state of the art in manufacturing science and technology.  Professional organizations provide the ideal forum for exchange of new ideas and approaches to exploring the boundaries of discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further the state of knowledge. \n\nExamples of ‘Objective Evidence’:\n•  Potential manufacturing science and technology gaps identified and documented \n\nQuestions (MRL Criteria in the form of a question):\n1. Have potential manufacturing science and technology gaps been identified?\n\nAdditional Considerations:  \n• Has manufacturing risk (i.e., availability of process, sources and materials) been considered?\n\nLessons Learned: \n• Historically we have not adequately addressed the manufacturing risk in this phase of technology development, so we must start addressing these risk issues earlier in the system life cycle."]
["$E$8",90,"Manufacturing technology requirements identified to address potential capability gaps for system concepts.","Purpose:  \nTo begin identifying manufacturing concepts to produce the product via experimentation and models and to use this information to understand the risk in pursuing these concepts. This will give decision makers insight into the risk and requirements for the new materials and processes and allow them to explore potential solutions for new concepts early in the S&T phase.  To effectively transition technology programs, one must address the manufacturing maturity of the prototypes being developed. Assessing the maturity of the manufacturing processes in S&T mandates that the R&D and acquisition communities work together, and a key part of that assessment is understanding manufacturing risk drivers and implications to production development strategy.  \n\nSources of Information:  \nThe S&T sponsors such as service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. At the beginning of technology development overall goals for cost, schedule and performance should be reviewed with customer(s) (users, S&T, and the acquisition community). It is critical that cost goals reflect manufacturing capabilities.\n\nExamples of ‘Objective Evidence’:\n•  Manufacturing technology requirements to address potential capability gaps for system concepts documented \n \nQuestions (MRL Criteria in the form of a question):\n1. Have Manufacturing technology requirements been identified to address potential capability gaps for system concepts?\n\nAdditional Considerations:  \n• Has manufacturing risk (i.e., availability of process, sources and materials) been considered?\n\nLessons Learned: \n• Historically we have not adequately addressed the manufacturing risk in this phase of technology development, so we must start addressing these risk issues earlier in the system life cycle."]
["$F$8",91,"Manufacturing technology development initiatives defined for preferred materiel solution. Manufacturing technology development requirements considered in the AoA.","Purpose:  \nTo start assessing the manufacturing processes of the proposed alternatives so as to identify advanced manufacturing technologies required to support those alternative designs. To use this process information to understand risk of the various alternatives as the program proceeds into Technology Maturation and Risk Reduction (TMRR). To quantify the risk of achieving program objectives and implement mitigation efforts. The program needs to understand the manufacturing risk of each proposed alternative and to reflect that risk in program planning efforts.  It is crucial to perform a manufacturing feasibility study for each proposed alternative and understand what, if any, new manufacturing technologies are required.  At the end of this phase, the alternative that best addresses user requirements will proceed into TMRR.  Manufacturing risk of selected alternative(s) will also be addressed in funding, planning, and other key program documentations (e.g., the Technology Development Strategy).\n\nSources of Information:\nThis data should come from the design and manufacturing departments.\n\nExamples of ‘Objective Evidence’:\n•  Pertinent Manufacturing Science (MS) and Advanced Manufacturing Technology requirements are identified and documented\n•  Analysis of Alternatives (AoA)\n\nQuestions (MRL Criteria in the form of a question):\n1. Have manufacturing technology development initiatives been defined for preferred materiel solution?\n2. Are manufacturing technology development requirements considered in the AoA?\n\nAdditional Considerations:  \n•Has manufacturing risk (i.e., availability of process, sources and materials) been considered?\n\nLessons Learned:\n•Historically we have not adequately addressed the manufacturing risk in this phase of technology development, so we must start addressing these risk issues earlier in the system life cycle.\n"]
["$G$8",92,"Required manufacturing technology development efforts initiated.","Purpose:  \nTo assess process capability by using data from prototype builds and similar production processes to assess if target costs are achievable.  To determine whether manufacturing development efforts or new technologies are required to achieve program objectives.  Using actual production data when performing these assessments can ensure that manufacturing risk of transitioning into production has been adequately assessed.  At the end of this phase the best alternatives will be selected to proceed into EMD.  Risk will also be understood for processes that require manufacturing technology development. \n\n\nSources of Information:  \nThis data should come from design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Documentation for availability of process, sources and material.\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have the required manufacturing technology development efforts been initiated?\nAdditional Considerations:  \n•Has manufacturing risk (i.e., availability of process, sources and materials) been considered?\n•The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms.\n\n\nLessons Learned: \n•Historically we have not adequately addressed the manufacturing risk in this phase of technology development, so we must start addressing these risk issues earlier in the system life cycle.\n"]
["$H$8",93,"Manufacturing technology efforts continuing.  Required manufacturing technology development solutions demonstrated in a production relevant environment. ","Purpose:  \nTo assess the progress of on-going manufacturing technology efforts and, using data generated in a production relevant environment, to determine whether new efforts are required. The program manager must understand the necessary manufacturing technology efforts and the associated risk to produce EMD units. \n\n\nSources of Information:  \nThis data should come from design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Manufacturing technology development plans\n•  Documentation that Manufacturing technology development solutions have been demonstrated in a production relevant environment\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Are the manufacturing technology development efforts underway and continuing? \n2.  Have the manufacturing technology development solutions required been successfully demonstrated in a production relevant environment?\n\n\nAdditional Considerations:  \n•  Are processes for fabrication of new or exotic materials clearly understood and available, or are they under development at this stage?\n•  The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms.\n\n\nLessons Learned: \n•  To adequately manage risk one must understand the progress of manufacturing technology development as the program enters EMD.\n"]
["$I$8",94,"Manufacturing technology efforts continuing.  Required manufacturing technology development solutions demonstrated in a production representative environment. ","Purpose:  \nTo ensure that adequate progress in our manufacturing technology efforts are being made by using demonstrated results from a production representative environment.  The program needs to verify that the needed manufacturing technology efforts are being supported and progressing satisfactorily.  It is essential to have confidence in the program’s ability to provide required manufacturing technology to support program objectives.\n\t\n\nSources of Information:  \nThis data should come from design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Manufacturing technology development efforts track to planned activities\n•Required manufacturing science and manufacturing technology development solutions have been demonstrated in a production representative environment\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are manufacturing technology efforts continuing and on schedule for pilot line implementation?\n2. Have the required manufacturing science and manufacturing technology development solutions been successfully demonstrated in a production representative environment? \n\n\n\nAdditional Considerations:  \n•Are processes for fabrication of new or exotic materials clearly understood and available, or are they under development at this stage?\n•The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms.\n\n\nLessons Learned: \n•To adequately manage risk one must understand the progress of manufacturing technology development as the program progresses in EMD.\n"]
["$J$8",95,"Primary manufacturing technology efforts concluding. Improvement efforts continuing.  Required manufacturing technology solutions validated on a pilot line.","Purpose:  \nTo ensure that adequate progress in the manufacturing technology efforts is being made by using demonstrated results from a pilot line environment.  To demonstrate that equipment and processes utilize mature technology and are adequate to achieve Low Rate Initial Production (LRIP) objectives.  Manufacturing equipment and processes should be based on new technology that is at a TRL of 7 or higher to support a pilot line for LRIP. The program must verify that all manufacturing technology efforts are supported and progressing satisfactory.  This will ensure that the production yield is sufficient to meet the program development schedule and that rework does not impact program cost and schedule.  A Pilot Line provides verifiable process capability/ performance results to compare with LRIP requirements.  Some of the data is used to determine whether tolerances can be met for Key Characteristics.  The program manager can then determine the likelihood of achieving the process capability to meet LRIP requirements and what risks remain going into LRIP. \n\n\nSources of Information:  \nThe majority of the results from the Pilot Line will come from the manufacturing/industrial engineering and quality departments. Key measures include process capability charts, cycle time, total manufacturing hours, scrap/rework percentages, first-pass-yield results, realization factors from actual versus standards, and change rates (design and process).  From these metrics a root cause analysis can indicate whether problems with the pilot production line are due to the introduction of immature technology or another manufacturing-related issue.\n\n\nExamples of ‘Objective Evidence’:\n•Manufacturing technology development plans\n•Primary manufacturing technology efforts documented and some improvement efforts continuing\n•Required manufacturing technology development solutions validated on a pilot line\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are the primary manufacturing technology efforts concluding? \n2. Are manufacturing improvement efforts continuing?\n3. Have the manufacturing technology solutions required been validated on a pilot line?\n\n\nAdditional Considerations:  \n•How faithful is the pilot line to a true production environment (personnel, equipment, processes, work instructions, etc.)?\n\n\nLessons Learned: \n•One of the important steps for assuring that adequate improvement in needed manufacturing technology is being made is to work with the contractor/ supplier on what truly represents a pilot production line for your programs. This effort should be done as soon as practical and if at all possible, before the award of the contract where this activity will be performed (e.g. EMD). \n•Manufacturing technology must be considered as part of the Technology Readiness Assessment (TRA).  Reviewing TRA results can provide indication that new manufacturing technology is being used.  The TRA may miss new manufacturing technology however, since TRAs tend to focus on system design.  \n•A review of information collected from earlier MRL assessments and findings is another important source of identifying new manufacturing technology insertion.  \n•The third and final way is to review each the pilot production lines for all parts of system development to determine if any new equipment or processes are being used.  For introduction of new equipment or processes, one must determine if it is based on introduction of new technology or reengineering of existing manufacturing systems—not based on new technology.  \n•A complete audit of the pilot production line equipment and processes can identify new technology being inserted.  \n•Once identified, focusing in on specific Statistical Process Control (SPC) measures may not be possible due to lot sizes (i.e., minimal data available), but other performance output measures can also be used to determine process capability (e.g., First Pass Yield, Scrap/Rework, Hours, Cycle Time, etc.). \n•Results that fall into expected norms can indicate that new technology is performing up to expectations, but results out of the expected band indicate possible influence of immature manufacturing technology and require a closer look.  \n•Root cause analysis of negative results and unexpected outputs may also indicate immature manufacturing technology.\n"]
["$K$8",96,"Manufacturing technology process improvements efforts initiated for FRP.","Purpose:  \nTo assess the progress of on-going manufacturing technology efforts and, using data generated in a production environment, to determine whether new efforts are required. The program manager must understand the necessary manufacturing technology efforts and the associated risk to produce Low Rate Initial Production (LRIP) units. \n\n\nSources of Information:  \nThis data should come from design and manufacturing departments.\n•  Manufacturing technology process improvement plans\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have manufacturing technology process improvement efforts been initiated for Full Rate Production (FRP)? \n\n\n\nAdditional Considerations:  \n• Look closely at yields and rates during LRIP to assure that you can effectively ramp up to FRP.\n\n\nLessons Learned: \n• Continuously assess production process and capabilities to support program requirements.\n• Implementing manufacturing technologies can be a critical tool for helping overcome shortfalls – especially in cost targets.  "]
["$L$8",97,"Manufacturing technology continuous process improvements ongoing. ","Purpose:  \nTo assess manufacturing technologies to improve efforts for FRP. The program constantly needs to assess options to improve the performance. Implement manufacturing technology programs effectively can be a major tool in this effort. \n\n\nSources of Information:  \nThis data should come from modeling/simulation, design, and manufacturing departments.\n \n\nExamples of ‘Objective Evidence’:\n•Manufacturing technology development plans\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are manufacturing technology continuous process improvements ongoing?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• A continuous improvement program and resultant elimination of waste (LEAN manufacturing) are critical indicators as to whether implementation of the manufacturing technologies has been truly successful.  "]
["$A$9",98,"B - Design   ","Design: Requires an understanding of the maturity and stability of the evolving system design and any related impact on manufacturing readiness.\n\nDesign includes producibility and maturity."]
["$B$9",99,"B.1 - Producibility Program"]
["$C$9",100,"Hypotheses developed for cause-effect relationships between technology variables and producibility.","Purpose: \n\nThe focus at this lowest level of manufacturing readiness is to address manufacturing shortfalls and opportunities to advance the state of the art in manufacturing.  Basic research (funded by budget) begins in the form of studies.  MRL assessments should only be used here to obtain knowledge that could be useful to make informed leadership decisions on future manufacturing risks or technologies to address when proceeding into Applied Research. Hypotheses are developed for cause-effect relationships between technology variables and producibility.  Within Basic Research, assessments focus on identifying technology options that can be applied to the manufacturing process to support delivery of higher performance and more affordable Warfighter solutions.\n\n\nSources of Information: \nThe S&T community, both S&T sponsors like the service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  Using hypotheses to obtain knowledge that would be useful to leadership to make informed decisions on which future cause-effect relationships between technology variables and producibility they may wish to address when proceeding into the Applied Research phase or to define manufacturing areas where more basic research needs to be done. The purpose of these assessments is to help transition capability to the warfighter more effectively and efficiently.    \n\n\nExamples of ‘Objective Evidence’:\n•  Hypotheses for cause-effect relationships between technology variables and producibility developed and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have hypotheses been developed for cause-effect relationships between technology variables and producibility? \n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$9",101,"Studies performed to test hypotheses regarding cause-effect relationships between technology variables and producibility. Elements identified which have a potential impact to producibility (i.e., materials, processes, capabilities, limitations).","Purpose: \nTo obtain knowledge that would be useful to leadership to make informed decisions on which future manufacturing risk areas or technologies they may wish to address when proceeding into the Applied Research phase or to define manufacturing areas where more basic research needs to be done. The purpose of these assessments is to help transition capability to the warfighter more effectively and efficiently.  MRL use within Basic Research focuses on identifying technology options that can be applied to the manufacturing process to support delivery of higher performance and more affordable warfighter solutions.  \n\nSources of Information: \nThe S&T community, both S&T sponsors like the service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further knowledge.\n\nExamples of ‘Objective Evidence’:\n•  Studies to test hypotheses regarding cause-effect relationships between technology variables and producibility performed and documented\n•  Elements which have a potential impact to producibility (i.e., materials, processes, capabilities, limitations) identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have studies been performed to test hypotheses regarding cause-effect relationships between technology variables and producibility? \n2. Have elements been identified which have a potential impact to producibility (i.e., materials, processes, capabilities, limitations)? \n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$E$9",102,"System concept elements evaluated for manufacturability and producibility using experiments, modeling, and simulation.","Purpose:   \nTo identify manufacturability and producibility for the alternative being considered and to use this information to understand manufacturing risk in managing the transition. To see some high level process charts to support this effort, and for new processes, have some data available based on a minimum of experimental data and modeling.  Manufacturability and producibility historically have not been evaluated until the product design is virtually frozen, thus incurring excessive product cost and product/process quality problems. Having to redesign the product to be affordable and capable of being produced in quantity is cost, quality and schedule prohibitive. \n\nSources of Information:  \nThe S&T community, both S&T sponsors like the service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further knowledge. \n\nExamples of ‘Objective Evidence’:\n•  Evaluation of relevant materials/processes for manufacturability using experiments/models is documented\nQuestions (MRL Criteria in the form of a question):\n1. Have relevant materials/processes been evaluated for manufacturability using experiments/models? \n\nAdditional Considerations: \n• Is there a need to evaluate/re-evaluate materials/processes characteristics for manufacturability and/or producibility issues?  \n• Have new manufacturing concepts and potential solutions been identified that could impact manufacturability/producibility?\n• Have new item gaps/risks impacting manufacturability and/or producibility been identified for key technologies, components, and/or key processes?\n• Have initial estimates of yields and rates based on experiments or state of the art been completed that could show potential manufacturability and/or producibility issue?\n\nLessons Learned: \n• Historically manufacturability and producibility issues are ignored early in S&T. \n• Producibility and manufacturability risks need to be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives."]
["$F$9",103,"Initial producibility and manufacturability assessments in selection of preferred materiel solution completed. Results considered in AoA documented in AS key components/technologies. ","B.1 - Producibility Program\n\n\nPurpose:\nTo ensure the producibility and manufacturability assessments have been performed and that the results will be used to identify the risk drivers in manufacturing proposed design alternatives.\nThe program must ensure that producibility and manufacturability risks are being adequately assessed and that the best alternatives will be considered.\n\nSources of Information:\nThis data should come from the design and manufacturing departments.\n\nExamples of ‘Objective Evidence’:\n•  Initial producibility & manufacturing assessments completed & documented\n•  Technology Development Strategy\n•  Analysis of Alternatives (AoA)\n\nQuestions (MRL Criteria in the form of a question):\n1. Are initial producibility and manufacturability assessments in selection of preferred materiel solution complete?\n2. Are manufacturing technology development requirements considered in the AoA?\n\nAdditional Considerations: \n•Is there a need to evaluate/re-evaluate materials/processes characteristics for manufacturability and/or producibility issues?  \n•Have new manufacturing concepts and potential solutions been identified that could impact manufacturability/producibility?\n•Have relevant materials/processes been evaluated for manufacturability using experimental results? \n•Have relevant materials/processes characteristics been identified for all new items? \n•Have new item gaps/risks impacting manufacturability and/or producibility been identified for key technologies, components, and/or key processes?\n•Have initial estimates of yields and rates based on experiments or state of the art been completed that could show potential manufacturability and/or producibility issue?\n\nLessons Learned: \n•Historically, manufacturability and producibility issues may not be adequately addressed early in acquisition. \n•Producibility and manufacturability risks of the alternatives need to be presented to the decision makers before down-selecting the preferred alternative design(s) to understand the overall risk of achieving program objectives.\n"]
["$G$9",104,"Producibility and manufacturability assessments of key technologies and components initiated.   Ongoing design trades consider manufacturing processes and industrial base capability constraints. Manufacturing processes assessed for capability to be tested and verified in production. Manufacturing processes assessed for influence on O&S. ","Purpose:\nTo obtain more fidelity in the producibility and manufacturability risk assessments by using prototype build data to focus on risk drivers.  At this stage the focus to identify manufacturing constraints should be at the component level for each prototype built, addressing specific producibility and manufacturability concerns (e.g., materials, labor, equipment, tooling/Special Test Equipment, setup, yield/scrap/rework, Work In Progress, and capability/capacity constraints).  It is essential for the program to address these issues when performing design trades to minimize risk and cost impact to the program. At the end of this phase the best available alternatives will be selected to proceed into EMD.\n\n\nSources of Information:  \nThis data should come from the design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Documented producibility & manufacturing assessments\n•  Trade studies\n•  Documented manufacturing process assessments\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have producibility & manufacturability assessments of key technologies and components been initiated?\n2.  Do the ongoing design trades include consideration of manufacturing processes and industrial base capability constraints? \n3. Have the manufacturing processes been assessed for capability to be tested and verified in production? \n4.  Have the manufacturing processes been assessed for their influence during the Operations and Support (O&S) phase?\n\n\nAdditional Considerations:\n•  Have all relevant materials/processes been evaluated for manufacturability and producibility in a production relevant environment?\n•  Can properly trained production personnel ultimately repeat the prototype work performed by engineers?\n•  Is the tooling and methods used, if applicable, be applicable ultimately in production?\n•  What are the ultimate manufacturing cost implications when in production? \n•  Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n\nLessons Learned: \n•  Historically manufacturability and producibility issues are ignored early in acquisition. \n•  Producibility and manufacturability risks of the alternatives need to be presented to the decision makers before down-selecting the preferred alternative design(s) to understand the overall risk of achieving program objectives.\n•  As the programs move from Milestone A into the TMRR phase there is a tendency for the manufacturing community to be insufficiently involved as a key team member.\n•  If manufacturability and producibility are continuously evaluated as the product technology design matures redesign costs and schedule delays can be avoided.\n"]
["$H$9",105,"Producibility assessments and producibility trade studies (performance vs. producibility) of key technologies/components completed.  Results used to shape AS, SEP, manufacturing and producibility plans, and planning for EMD or technology insertion programs.  Preliminary design choices assessed against manufacturing processes and industrial base capability constraints.  Producibility enhancement efforts (i.e., DFM, DFA, etc.) initiated.","Purpose:\nTo assess producibility data from building prototypes in a production relevant environment, compare it to targets and assess manufacturing risk of meeting future requirements. To use this data to ensure that the Acquisition Strategy, Systems Engineering Plan, Manufacturing and Producibility plans, and planning for EMD apply appropriate risk mitigation and make appropriate trades in the preliminary design process. At this phase of the program the design must be produced in a production-relevant environment, and producibility results must be used in design trade-offs and program planning efforts. This is the opportune time to make design changes without negatively impacting program cost and schedule.  Addressing producibility at this stage of the acquisition cycle is critical to achieving affordability objectives.\n\n\nSources of Information:\nThis data should come from engineering design, and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Producibility assessments \n•  Trade studies\n•  Manufacturing Plan\n•  Producibility enhancement efforts\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have assessments and trade studies regarding performance vs. producibility for key technologies and/or components been completed?\n2.  Have the results of the producibility assessments and trade studies been used to shape the Acquisition Strategy, the Systems Engineering Plan, the plans for manufacturing and producibility plans, and planning for the Engineering and Manufacturing Development phase or for technology insertion programs?  \n3.  Have the preliminary design choices been assessed against the constraints of the manufacturing processes and the capability of the industrial base?\n4.  Have efforts to enhance the producibility (i.e., DFM, DFA, etc.) been initiated? \n\n\nAdditional Considerations: \n•  Are the tooling and methods used applicable ultimately in production?\n•  Have the manufacturing processes been assessed for capability to test and verify in production and their influence on Operations and Support (O&S)?\n•  Have all relevant materials/processes been evaluated for manufacturability and producibility in a production relevant environment? \n•  Can properly trained production personnel repeat the prototype work performed by engineers? \n•  Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n•  What are the ultimate manufacturing cost implications when in production? \n\n\nLessons Learned:\n•  Historically manufacturability and producibility issues are ignored early in acquisition. \n•  Producibility and manufacturability risks of the alternatives need to be presented to the decision makers before moving into EMD.\n•  This is the opportune time to make design changes without negatively impacting program cost and schedule.\n"]
["$I$9",106,"Detailed producibility trade studies using knowledge of key design characteristics and related manufacturing process capability completed.   Producibility enhancement efforts (i.e., DFM, DFA, etc.) ongoing for optimized integrated system.  Manufacturing processes re-assessed as needed for capability to be tested and verified. Manufacturing processes re-assessed as needed for potential influence on O&S.","Purpose:\nDemonstrate process capability in a production representative environment and to obtain fidelity in assessing producibility of the product by using data from building EMD units. Data generated from this assessment will be used to make design choices and do make-or-buy, capacity, process capability, sources, quality, yield/rate, and variability assessments.  This information is used to determine what will be required to meet Production and Operations and Support requirements and identify needed producibility improvements.  At this phase of a program the item/system must be manufactured in a production representative environment.  Process capability and design producibility must be assessed against program requirements.  It is essential to use actual manufacturing data while performing these assessments to truly identify producibility and manufacturing risk entering LRIP and subsequently transitioning to Full-Rate Production.\n\n\nSources of Information:\nThis data should come from design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Producibility assessments \n•Trade studies\n•Producibility enhancement efforts\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have detailed producibility trade studies using knowledge of key design characteristics and the related manufacturing process capability been completed?\n2. Are producibility enhancement efforts (i.e. Design for Manufacturing, Design for Assembly, etc.) ongoing for the optimized integrated system? \n3. Have manufacturing processes been re-assessed as needed for capability to be tested and verified?\n4. Have manufacturing processes been re-assessed as needed for potential influence on the Operations and Support phase?\n\n\n\nAdditional Considerations: \n•Have the manufacturing processes been assessed for capability to test and verify in production and their influence on Operations and Support (O&S)?\n•Have all relevant materials/processes been evaluated for manufacturability and producibility in a production representative environment? \n•Can properly trained production personnel repeat the prototype work performed by engineers? \n•Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n•What are the ultimate manufacturing cost implications when in production? \n\n\nLessons Learned:\n•It is crucial to continually address the producibility of hardware by assessing demonstrated process capability and, as more data becomes available, see if it will to meet production requirements.\n•This can be a good time to make design changes with minimal negative impact to program cost and schedule.\n"]
["$J$9",107,"Producibility improvements implemented on system. Known producibility risks and issues managed for LRIP. ","Purpose:\nDemonstrate the producibility in a pilot line environment and obtain more fidelity assessing producibility/process capability by using data from building EMD units.  Use this information to ensure that all producibility issues have been resolved, that LRIP requirements will be met and to make producibility improvements on the system. At this phase of the system life-cycle the product should be built in a pilot line environment.  The Data generated will be used to assess the producibility of the product against program requirements. It is essential to use actual manufacturing data while performing these assessments to truly identify producibility and manufacturing risk entering LRIP and subsequently transitioning to Full-Rate Production.\n\nSources of Information:\nThis data should come from design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Producibility improvements implemented on the system \n•Documentation verifying that known producibility issues have been resolved and pose no significant risk for Low Rate Initial Production (LRIP)\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have the producibility enhancement efforts (improvements) been implemented on system (for pilot line)?\n2. Are all known producibility risks and issues being managed for Low Rate Initial Production (LRIP)?\n\n\nAdditional Considerations: \n•Have the manufacturing processes been assessed for capability to test and verify in production and their influence on Operations and Support (O&S)?\n•Have all relevant materials/processes been evaluated for manufacturability and producibility in a pilot line environment? \n•Can properly trained production personnel repeat the prototype work performed by development personnel? \n•Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n•What are the ultimate manufacturing cost implications when in production? \n\nLessons Learned:\n•It is important to continue to address the producibility of hardware by assessing demonstrated process capability and, as more data becomes available, see if it will meet production requirements.\n"]
["$K$9",108,"Prior producibility improvements analyzed for effectiveness during LRIP.  Producibility risks and issues discovered in LRIP for FRP. ","Purpose:  \nTo use this data to ensure that the Acquisition Strategy, Systems Engineering Plan, Manufacturing and Producibility plans, and planning for Low Rate Initial Production (LRIP) apply appropriate risk mitigation and make appropriate trades in the design process. At this phase of the program the design must be produced in a production environment. Producibility results may still be used in design trade-offs and program planning efforts, but these changes could negatively impact program cost and schedule.  \n\n\nSources of Information:\nThis data should come from engineering design, and manufacturing departments.\nExamples of ‘Objective Evidence’:\n•  Producibility assessments \n•  Trade studies\n•  Producibility enhancement efforts\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have prior producibility enhancement efforts (improvements) been analyzed for effectiveness during Low Rate Initial Production (LRIP)?\n2. Are any additional producibility issues and risks discovered in LRIP being managed for FRP?  \n\n \nAdditional Considerations:  \n• What are the ultimate manufacturing cost implications when in production? \n• Are producibility and process capability assessments ongoing to identify possible cost reductions?\n\n\nLessons Learned: \n• Continually update assessments of product producibility as more data becomes available and use this data to assess whether program objectives can be met. "]
["$L$9",109,"Design producibility improvements demonstrated in FRP.  Process producibility improvements ongoing.  All modifications, upgrades, Diminishing Mfg Sources & Material Shortages (DMSMS) and other changes assessed for producibility.  ","Purpose:  \nDemonstrated producibility is continuing to be used to address all configuration changes to meet FRP objectives and improve cost. The program should maintain an effective producibility program addressing all configuration changes to ensure program objectives are being addressed.\n\nSources of Information:  \nThis data should come from cost, engineering design, and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Producibility assessments \n•Demonstration of producibility improvements in FRP\n•Producibility enhancement efforts\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have design producibility improvements been demonstrated in Full Rate Production (FRP)? \n2. Are process producibility improvements (continuous improvement enhancement efforts) ongoing?\n3. Are all modifications, upgrades, Diminishing Manufacturing Sources & Material Shortages mitigation efforts and other changes being assessed for producibility?\n\n\n\nAdditional Considerations:  \n• Are producibility and process capability assessments ongoing to identify possible cost reductions?\n\n\nLessons Learned: \n• Continually update assessments of product producibility as more data becomes available and use this data to assess whether program objectives can be met."]
["$A$10",110]
["$B$10",111,"B.2 - Design Maturity"]
["$C$10",112,"Current capability deficiencies and gaps identified.","Purpose: \n\nThe focus at this lowest level of manufacturing readiness is to address manufacturing shortfalls and opportunities to advance the state of the art in manufacturing.  Basic research (funded by budget) begins in the form of studies.  MRL assessments should only be used here to obtain knowledge that could be useful to make informed leadership decisions on future manufacturing risks or technologies to address when proceeding into Applied Research. The purpose of manufacturing assessments in S&T is to transition capability to the warfighter more effectively and efficiently.  Within Basic Research, assessments focus on identifying technology options that can be applied to the manufacturing process to support delivery of higher performance and more affordable Warfighter solutions.\n\n\nSources of Information: \n\nThe S&T community, both S&T sponsors like the service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further knowledge. \n   \n\nExamples of ‘Objective Evidence’:\n•  Current capability deficiencies and gaps identified and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have current capability deficiencies and gaps been identified?\n\t\n\nAdditional Considerations:\n• What are the major impediments to manufacturing with new materials or achieving streamlined manufacturing processes producing more affordable products?\n• What new Warfighter capabilities are being held back due to manufacturing costs or full scale productization solutions?\n• Where are we reaching the point of diminishing returns in manufacturing efficiency improvements due to limitations in current processes or availability of materials where alternate methods may offer new solutions?\n• Who are the major investment supporters for manufacturing S&T and what programs are they funding?\n• What are the new frontiers in manufacturing S&T where breakthroughs are likely to make a major impact?\n\n\nLessons Learned: \n• A community of interest approach to generating collaboration between S&T, requirements, acquisition and manufacturing is essential in communicating manufacturing challenges for future exploration by the research community."]
["$D$10",113,"Analyses performed to evaluate the feasibility of potential solutions to address capability gaps. ","Purpose: \nTo obtain knowledge that would be useful to leadership to make informed decisions on which future manufacturing risk areas or technologies they may wish to address when proceeding into the Applied Research phase or to define manufacturing areas where more basic research needs to be done. The purpose of these assessments is to help transition capability to the warfighter more effectively and efficiently.  MRL use within Basic Research focuses on identifying technology options that can be applied to the manufacturing process to support delivery of higher performance and more affordable warfighter solutions.  \n\nSources of Information: \nThe S&T community, both S&T sponsors like the service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further knowledge.\n \nExamples of ‘Objective Evidence’:\n•  Analyses to evaluate the feasibility of potential solutions to address capability gaps performed and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have analyses been performed to evaluate the feasibility of potential solutions to address capability gaps?\n  \nAdditional Considerations:\n• What are the major impediments to manufacturing with new materials or achieving streamlined manufacturing processes producing more affordable products?\n• What new Warfighter capabilities are being held back due to manufacturing costs or full scale productization solutions?\n• Where are we reaching the point of diminishing returns in manufacturing efficiency improvements due to limitations in current processes or availability of materials where alternate methods may offer new solutions?\n• Who are the major investment supporters for manufacturing S&T and what programs are they funding?\n• What are the new frontiers in manufacturing S&T where breakthroughs are likely to make a major impact?\n• What new manufacturing concepts are being explored and how do they apply to DOD manufacturing obstacles?\n\nLessons Learned: \n• A community of interest approach to generating collaboration between S&T, requirements, acquisition and manufacturing is essential in communicating manufacturing challenges for future exploration by the research community."]
["$E$10",114,"High level performance, lifecycle, and technical requirements defined and evaluated for system concepts. Trade-offs in design options based on experiments and initial MOEs.","Purpose:\nTo obtain knowledge that would be useful to leadership to make informed decisions on which future manufacturing risks or technologies they may wish to address when proceeding into the Applied Research phase, or to define manufacturing areas where more basic research needs to be done. The purpose of manufacturing assessments during R&D is to help transition capability to the warfighter more effectively and efficiently.  MRL use within Basic Research focuses on identifying technology options that can be applied to the manufacturing process to support delivery of higher performance and more affordable warfighter solutions.  \n\nSources of Information: \nInformation comes from the Program Office that is establishing the program, either in the PEO, SYSCOM or Warfare Center/DoD laboratory (i.e., the organization putting forth the initial concept and establishing the capability requirement).  The AOA will identify potential solutions that can lead to potential design approaches.  From here manufacturing needs can be deduced.\n   \nExamples of ‘Objective Evidence’:\n•  High level performance, lifecycle, and technical requirements are documented\n•  High level performance, lifecycle, and technical requirements evaluated for system concepts and documented\n•  Trade-offs in design options are based on experiments and initial MOEs and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have high level performance, lifecycle, and technical requirements been defined?\n2. Have high level performance, lifecycle, and technical requirements been evaluated for system concepts? \n3. Are trade-offs in design options based on experiments and initial MOEs?\n\nAdditional Considerations:\n• What are the envisioned manufacturing challenges for the potential technologic solutions being considered for this capability?\n• Are new manufacturing technologies potentially required to address these challenges?\n• Are there new manufacturing processes needed to address the anticipated production requirements?\n• Who are the major investment supporters for manufacturing S&T and what programs are they funding?\n• What are the new frontiers in manufacturing S&T where breakthroughs are likely to make a major impact?\n\nLessons Learned: \n• Identifying new material usage or manufacturing needs or techniques that extend beyond the current production boundaries early is essential to making smart design decisions and avoiding manufacturing risk.\n• Early investment in manufacturing technology or process development is essential to avoiding major cost, schedule and performance issues during follow on acquisition phases of the program. "]
["$F$10",115,"Form, fit, and function constraints, and manufacturing capabilities identified for preferred systems concept. SEP and T&E Strategy recognize the need for the establishment/validation of manufacturing capability and management of manufacturing risk for the product lifecycle. Initial KPPs identified for preferred systems concept. System characteristics and measures to support required capabilities identified.","Purpose: \nTo ensure manufacturability, producibility, and quality processes are in place and sufficient to produce technology demonstrators.  To identify manufacturing risks for building prototypes and ensure mitigation plans are in place.  To identify manufacturing cost drivers and establish target cost objectives.  Initial producibility assessments of design concepts have been completed.  Key design performance parameters have been identified as well as special tooling, facilities, material handling and skills required.  A manufacturing feasibility assessment at his time helps a program to: (1) Better understand the risk; (2) begin risk mitigation efforts, and (3) provides critical information to accurately reflect the financial risk in both estimating and funding processes. It is important before selecting any potential solution that manufacturing feasibility/ readiness be evaluated to understand the risk of achieving the cost and schedule for any proposed approach.\n\n\nSources of Information: \nThe majority of information will come from the Analysis of Alternatives, the Initial Technical Review and the Alternative Systems Review. \n\n\nExamples of ‘Objective Evidence’:\n•  Form, fit, and function constraints, and manufacturing capabilities, identified and documented for the preferred systems concept\n•  Manufacturing capability and management of manufacturing risk documented in Systems Engineering Plan (SEP)\n•  Manufacturing capability and management of manufacturing risk documented in the Test and Evaluation (T&E) Strategy \n•  Key Performance Parameters (KPPs) identified and documented for the preferred systems concept\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have form, fit, and function constraints, and manufacturing capabilities, been identified for the preferred systems concept?\n2. Does the SEP and T&E Strategy recognize the need for the establishment/validation of manufacturing capability and management of manufacturing risk for the product lifecycle?\n3. Have initial Key Performance Parameters (KPPs) been identified for the preferred systems concept?\n4. Have system characteristics and measures to support required capabilities been identified?\n\n\nAdditional Considerations:\n• Is the item design novel, or does it contain nonstandard dimensions or tolerances or arrangements?\n• Are there manufacturing technologies or processes not currently available that will lead to risk in advanced technology development?\n• Does DOD or industry need to make investments to create new industrial capabilities to support a robust design?\n• What is the risk of industry not being able to provide new program design at planned cost and schedule?\n\n\nLessons Learned: \n• Identification of critical manufacturing technology and processes early on is critical to making low risk design trade-offs.  \n• If there are major manufacturing obstacles due to unavailable or immature manufacturing technology, alternative design choices should be considered.\n"]
["$G$10",116,"Lower level performance requirements sufficient to proceed to preliminary design.  All enabling/critical technologies and components identified and the product lifecycle considered.  Evaluation of the design for KCs initiated. Product data required for prototype component manufacturing released.","Purpose: \nTo identify and assess the manufacturing risk of proposed alternatives, to begin maturing the manufacturing processes to reduce risk, and assess the degree to which the system is adequately demonstrated in a relevant environment. The manufacturing risk identified in this phase must be addressed in EMD through risk mitigation efforts.  Risk mitigation cost must also be reflected in cost estimates.  The first step is to require the program to generate the design data needed for a manufacturing risk assessment and to introduce design related activities to mature manufacturing processes.  The assessment process will define critical design activities needed to reach a target level of manufacturing maturity. \n\n\nSources of Information: \nData from AoAs, Assessments of Manufacturing Readiness, the Initial Technical Review (ITR), the Alternative Systems Review (ASR) and Technology Readiness Assessments (TRA) can help the program manager understand the degree of manufacturing risk inherent in each proposed alternative. \n\n\nExamples of ‘Objective Evidence’:\n•  Lower level performance requirements documented\n•  Enabling/critical technologies and components documentation\n•  Documentation of the evaluation of design Key Characteristics (KCs)\n•  Documentation of product data required for prototype component manufacturing released\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have lower level performance requirements (e.g., those needed to meet initial KPPs) been analyzed as sufficient to proceed to the preliminary design?  \n2.  Have all enabling and/or critical technologies and components (e.g., those needed to meet performance requirements) been identified? \n3.  Has the product lifecycle been considered in identifying enabling and/or critical technologies and components?\n4.  Have the evaluation of the design for Key Characteristics (KCs) been initiated? \n5.  Has the required product data to manufacture prototype components been released?\n\n\nAdditional Considerations: \n•  Are there materials which have not been demonstrated in similar products or manufacturing processes? \n•  Is this item a driver that significantly impacts life-cycle cost (development, unit, or operations and/or support costs)?  Is the technology new with high cost uncertainty?   \n•  Is the item design novel or does it contain nonstandard dimensions, tolerances or arrangements? \n•  Will the item require the use of manufacturing technology, processes, inspection, or capabilities that are unproven in the current environment? \n•  Does the item have historical/anticipated yield or quality issues? \n•  Are all required manufacturing processes and techniques currently available or are some still to be defined?\n•  What is the probability of meeting the delivery date (e.g., for prototypes)?\n•  What are the design producibility risks?\n•  What is the potential impact of critical and long-lead time material?\n•  Are there manufacturing capability, and cost/schedule impact analyses to support trade-offs among alternatives?\n\n\nLessons Learned: \n•  Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control.  \n•  There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n\n"]
["$H$10",117,"System allocated baseline established.  Product requirements and features are well enough defined to support PDR. Product data essential for subsystem/system prototyping has been released, and all enabling/critical components have been prototyped. Preliminary KCs for the design identified and mitigation plans initiated.","Purpose: \nTo assess the manufacturing maturity of the program using MRL criteria to ensure recommended activities have been accomplished before proceeding into EMD, and to use design data created up to this point to assess the manufacturing risk and determine whether manufacturing processes will fall within the program’s cost and schedule objectives.  To develop risk mitigation plans to minimize manufacturing risk in EMD.  Demonstrating critical manufacturing processes in a production relevant environment is critical to assessing manufacturing risk and obtaining confidence you can achieve program cost, schedule and performance requirements before proceeding into EMD.  To do this, the design must be matured for critical manufacturing technology areas.  Once determined, pilot line production capability can be established to start manufacturing critical parts and components.  The emphasis is on addressing higher risk areas (e.g., more advanced technologies and newer manufacturing capabilities). During this phase it is essential that the contractor demonstrate the capability to build the product, or a similar product, in the facility that will be used during production. \n\n  \nSources of Information: \nData from Lifecycle Cost Estimates, the Systems Requirements Review (SRR), the Systems Functional Review (SFR), the Cost Analysis Requirements Description (CARD), the Preliminary Design Review (PDR), the Integrated Baseline Review (IBR) and Technology Readiness Assessments (TRA) can help the program manager understand the degree of manufacturing risk inherent in each proposed alternative.\n\n\n\nExamples of ‘Objective Evidence’:\n•  System Allocated Baseline\n•  Defined product requirements and features\n•  Prototype product data released\n•  Demonstration documentation\n•  Preliminary KCs documented and mitigation plans\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Has the allocated baseline for the system been established?\n2.  Are the product requirements and features well enough defined to support Preliminary Design Review (PDR)?\n3.  Has the required product data essential for manufacture of subsystem prototypes and/or the system prototypes been released?\n4.  Have all enabling and/or critical components been prototyped?\n5. Have all preliminary Key Characteristics (KCs) for the design been identified?\n6. Have mitigation plans for Key Characteristics (KCs) been initiated?\n\n\n\nAdditional Considerations:\n•  Are there materials which have not been demonstrated in similar products or manufacturing processes? \n•  Is this item a driver that significantly impacts life-cycle cost (development, unit, or operations and/or support costs)?  Is the technology new with high cost uncertainty?   \n•  Is the item design novel or does it contain nonstandard dimensions, tolerances or arrangements? \n•  Will the item require the use of manufacturing technology, processes, inspection, or capabilities that are unproven in the current environment? \n•  Does the item have historical/anticipated yield or quality issues? \n•  Are all required manufacturing processes and techniques currently available or are some still to be defined?\n•  What is the probability of meeting the delivery date (e.g., for prototypes)?\n•  What are the design producibility risks?\n•  What is the potential impact of critical and long-lead time material?\n•  Are there manufacturing capability, cost and schedule impact analyses to support trade-offs among alternatives?\n\n\nLessons Learned: \n•  A production relevant environment is different for different products, which is often dependent on rates and quantities of items being produced and the complexity of those items.  For example, a production relevant environment for a satellite is different than that used in building prototype vehicles.\n•  Statistical Process Control is not a requirement for all programs.  Rates and Quantities play a major role in the decision to manage product quality using SPC.\n•  Many contractors do not understand the concept of Key Characteristics and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control.  \n•  There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n"]
["$I$10",118,"Product design and features are well enough defined to support CDR, even though design change traffic may be significant. All product data essential for component manufacturing released. Potential KC risks and issues identified with mitigation plans in place.","Purpose: \nTo reduce system level risk.  EMD typically includes the demonstration of production prototype articles or engineering development models. EMD results in the completion of the design, or at least the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality.   \n\n\nSources of Information: \nData from the Preliminary Design Review (PDR), the Cost Analysis Requirements Description (CARD), the program schedule, the Integrated Baseline Review (IBR), Technology Readiness Assessments (TRA) and Assessments of Manufacturing Readiness can help the program manager understand the degree of manufacturing risk inherent in each proposed alternative.\n\n\nExamples of ‘Objective Evidence’:\n•Preliminary Design Review (PDR) Data\n•Integrated Baseline\n•Technology Readiness Assessment (TRA)\n•MRL Assessment (MRA)\n•Defined product design and features\n•All product data essential for component manufacturing released\n•Potential Key Characteristic (KC) risk issues identified\n•Preliminary Key Characteristic (KC) issues defined and mitigation plans are in development\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are the product design and features well enough defined to support Critical Design Review?\n2. Has the impact of design change traffic on design maturity been evaluated as a risk or issue?\n3. Has all of the product data essential for manufacture of products at the component level been released?\n4. Have risks and issues to Key Characteristics been identified?\n5. Are identified Key Characteristic (KC) risk mitigation plans in place?\n\n\nAdditional Considerations:\n•Is system design sufficient and stable enough to successfully pass CDR?\n•Are all technologies mature enough for production?\n•Are the design risks known and manageable?\n•Industrial base viability\n•Probability of meeting the delivery date (e.g., for qualification units)\n•Process maturity\n•Manufacturing costs\n•Supply chain management\n•Quality management\n•Facilities\n•Manufacturing skills availability\n\n\nLessons Learned: \n•A production representative environment is different for different products, which is often dependent on rates and quantities of items being produced, and the complexity of those items.  For example, a production representative environment for a satellite is different than that used in building prototype vehicles.\n•Statistical Process Control is not a requirement for all programs.  Rates and Quantities play a major role in the decision to manage product quality using SPC.\n•Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control.  \n•There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n"]
["$J$10",119,"Detailed design of product features and interfaces completed. All product data essential for system manufacturing released.  Design change traffic does not significantly impact LRIP. KCs are attainable based upon pilot line demonstrations.","Purpose: \nTo assure known producibility risks pose no significant challenges for low rate production and that cost model and yield and rate analyses have been updated with pilot line results.  The Engineering and Manufacturing Development (EMD) phase results in the completion of the design, at least the near completion of the design.   Major design issues are well within program constraints and should have limited impact on product quality. The MRL 8 evaluation is used to support a Milestone C decision to begin Low Rate Initial Production (LRIP).   \n\n\nSources of Information: \nData from the Production Readiness Review (PRR), the Critical Design Review (CDR), the Capability Production Document (CPD) and Assessments of Manufacturing Readiness will give the program manager confidence that he has addressed the manufacturing risk prior to entering into LRIP.\n\n\n\nExamples of ‘Objective Evidence’:\n•Critical Design Review (CDR) Data \n•Detailed design of product features and interfaces complete \n•All product data essential for system manufacturing released \n•Design change traffic has minimal impact on Low Rate Initial Production (LRIP)\n•Key Characteristics attainable based upon pilot line demonstrations \n•MRL Assessment (MRA)\n•Demonstration documentation\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Is the detailed design of product features and interfaces complete?\n2. Has all the product data essential for manufacture of the system been released?\n3. Does the design change traffic significantly impact Low Rate Initial Production (LRIP)?\n4. Based on pilot line demonstration and evaluation, are all Key Characteristics attainable?\n\n\n\nAdditional Considerations:\n•Has the system product design baseline been established and documented to enable hardware fabrication and software coding to proceed with proper configuration management? \n•Are adequate processes and metrics in place for the program to succeed? \n•Are the design risks known and manageable? \n•Are all technologies mature enough for production? \n•Is the detailed design producible within the production budget? \n•Is detail design complete and stable enough to enter low rate production? \n•Have manufacturing processes been demonstrated and proven in a pilot line environment? \n•Have all producibility trade studies and risk assessments been completed? \n•Is the production cost model based upon the stable detailed design and been validated? \n•Are the production facilities ready and required workers trained?\n•Is the program schedule executable (technical/cost risks)?\n•Is the program properly staffed?\n•Does the system or subsystem have Information Assurance (IA) Certification and Accreditation approval from the system DAA?  Is a signed authorization memo available?  If the answer is no, contact the program’s Information Assurance POC to discuss potential risks.\n\n\n\nLessons Learned: \n•A pilot line environment is different for different products, which is often dependent on rates and quantities of items being produced, and the complexity of those items.  For example, a pilot line environment for a satellite is different than that used in building prototype vehicles.\n•Statistical Process Control is not a requirement for all programs.  Rates and Quantities play a major role in the decision to manage product quality using SPC.\n•Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control.  \n•There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n•Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area.  \n•Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality.  Suppliers may not understand the application of their products into the overall system.  The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II.\n"]
["$K$10",120,"Major product design features and configuration are stable. System design has been validated through operational testing of LRIP items. PCA or equivalent complete as necessary.  Design change traffic is limited.  All KCs are controlled in LRIP to appropriate quality levels.","Purpose: \nTo assess the manufacturing maturity of the program using MRL criteria to ensure recommended activities have been accomplished before proceeding into FRP, and to use design data created up to this point to assess the manufacturing risk and determine whether manufacturing processes will fall within the program’s cost and schedule objectives.  To develop risk mitigation plans to minimize manufacturing risk in FRP.  Demonstrating critical manufacturing processes in a production environment is critical to assessing manufacturing risk and obtaining confidence you can achieve program cost, schedule and performance requirements before proceeding into FRP.  To do this, the design must be matured for critical manufacturing technology areas.  Once determined, LRIP capability can be established to start manufacturing critical parts and components.  The emphasis is on addressing higher risk areas (e.g., more advanced technologies and newer manufacturing capabilities). \n\n\nSources of Information: \nInformation can be obtained from Detailed Design Packages and Developmental and Operational Test and Evaluation Reports.\nExamples of ‘Objective Evidence’:\n•  System configuration management shows stability of changes\n•  System design validated through operational testing \n•  Documentation for Physical Configuration Audit (PCA) or equivalent complete\n•  Design change traffic limited to minor configuration changes \n•  Documented control of Key Characteristics (KCs) in Low Rate Initial Production (LRIP) to appropriate quality levels\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are the major product design features and the configuration stable?\n2. Has operational testing of production products validated the system design?\n3. Has a Physical Configuration Audit (or equivalent) determined conformance of the as-built product to the documented design and verified the product baseline?\n4. Is the design change traffic limited?\n5. Are all Key Characteristics controlled in Low Rate Initial Production (LRIP) to production quality levels?\n\n\nAdditional Considerations:\n• Are there unresolved design changes that need to be implemented, and if so, what are the impacts on manufacturing?\n• Are the manufacturing processes under control?\n• Can the system be reliably produced?\n• Are critical processes under control?\n• Have key issues been resolved with appropriate changes to the Product Baseline?\n• Have necessary statutory and regulatory provisions been met?\n• Have all risks been identified, including appropriate mitigation plans?\n\n\nLessons Learned: \n• Statistical Process Control is not a requirement for all programs.  Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control.  \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
["$L$10",121,"Product design is stable.  Design changes are few and generally limited to those required for continuous improvement or in reaction to obsolescence.  All KCs are controlled in FRP to appropriate quality levels.","Purpose: \nTo ensure all materials, tooling, inspection and test equipment, facilities and manpower are in place and have met full rate production requirements, that rate production unit costs meet goals, and funding is sufficient for production at required rates. Lean practices are well established and continuous process improvements are ongoing. The FRP decision requires that manufacturing risk is understood and that the manufacturing processes for the system be capable, in control, and affordable. Prior to the FRP decision, a manufacturing readiness assessment should be conducted to ensure any outstanding risks will not impact the programs ability to deliver FRP requirements.  \n\n\nSources of Information: \nInformation can be obtained from Detailed Design Packages and Developmental and Operational Test and Evaluation Reports.\n\n\nExamples of ‘Objective Evidence’:\n•Evidence that product design is stable with few design changes\n•Evidence that all Key Characteristics are controlled in FRP to appropriate quality levels \n\n\nQuestions (MRL Criteria in the form of a question):\n1. Is the product design stable?\n2. Are design changes limited to continuous improvement or to obsolescence?\n3. Are all Key Characteristics (KCs) controlled in Full Rate Production (FRP) to planned quality levels?\n\n\n\nAdditional Considerations:\n• Are there unresolved design changes that need to be implemented, and if so, what are the impacts on manufacturing?\n• Are the manufacturing processes under control?\n• Can the system be reliably produced?\n• Are critical processes under control?\n• Have key issues been resolved with appropriate changes to the Product Baseline?\n• Have necessary statutory and regulatory provisions been met?\n• Have all risks been identified, including appropriate mitigation plans?\n\n\nLessons Learned: \n• Assessments of manufacturing readiness may be used to capture manufacturing product documentation. \n• It is a best practice to incorporate the preservation of such manufacturing product technical data packages in the Data Management Strategy."]
["$A$11",122,"C - Cost & Funding","Cost and Funding: Requires an analysis of the adequacy of funding to achieve target manufacturing maturity levels. Examines the risk associated with reaching manufacturing cost targets.\n\nCost and Funding includes production cost knowledge (cost modeling), cost analysis, and manufacturing investment budget."]
["$B$11",123,"C.1 - Production Cost Knowledge (Cost modeling)"]
["$C$11",124,"Hypotheses developed regarding technology impact on affordability.","Purpose:  \nTo begin identifying manufacturing cost implications, and to use this information to understand the risk in pursuing a given approach. To allow decision makers insight into the cost risk and start exploring mitigation options to address identified cost risks.  Developers need to begin the process of identifying manufacturing cost implications for any approach being considered.  \n\n\nSources of Information:  \nThe S&T community, including service laboratories and research organizations in academia and industry provide information for technology impact on affordability.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further knowledge.    \n\n\nExamples of ‘Objective Evidence’:\n•  Hypotheses regarding technology impact on affordability developed and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have hypotheses been developed regarding technology impact on affordability?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$11",125,"Cost model approach defined.","Purpose:  \nTo identify approaches that will be used in later phases to assess cost and understand cost risk in pursuing the design. Provide decision makers insight into the cost risk and associated mitigation options.  \n\n\nSources of Information:  \nThe S&T community, including service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further knowledge. \n \n\nExamples of ‘Objective Evidence’:\n•  Cost elements identified and documented\n \nQuestions (MRL Criteria in the form of a question):\n1. Have cost elements been identified?\n  \nAdditional Considerations:  \n• Will new materials or product development concepts most likely be required?\n• Is a new technology or process available to process the material?\n• Scrap, rework and repair from low-yield processes must be considered in the total product cost estimate.\n• Approaches could include goals on how cost models will be used in high level trades (architectural concept selection & optimization), system level design, and flow-down to components (e.g., QFD with cost model weights, Pugh Concept Selection (PCS), CAIV, etc.).\n• Approaches could include how the life-cycle cost of the product will modeled including goal-setting (parameters could include R&D, transition, ramp/learning, sustainment, hidden factory, logistics, supplier, after-market, repair and disposal costs).\n\nLessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must start addressing what model we will use to assess the manufacturing cost earlier in the acquisition process. \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• In this early development phase it is extremely difficult to gain real manufacturing expertise experience to understand potentially critical manufacturing cost issues unless reviewers have had significant experience with Integrated Product and Process Development versus “silo” product development.  \n• Cost estimates at this stage of the product life-cycle should be considered as ROM estimates and be open to adjustment as the design is firmed up."]
["$E$11",126,"Manufacturing cost estimates for system concepts developed. Initial cost models developed which include high-level process steps and materials.","Purpose:  \nTo identify approaches that will be used in later phases to assess cost and understand cost risk in pursuing the design. Provide decision makers insight into the cost risk and associated mitigation options. To begin identifying cost targets and risk for the alternative being considered, and use this information to understand the cost risk of managing the transition.  Put manufacturing cost on an equal footing with technical requirements. The project must project production cost by setting initial cost targets and identifying cost risks of producing alternative designs.  At the end of Pre-MSA the best available alternatives will be selected to address entrance criteria for MSA. \n\nSources of Information:  \nThe S&T community, including service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Internal market analyses and customer input will also help gauge what is an acceptable cost for the product.\n\nExamples of ‘Objective Evidence’:\n•  Manufacturing cost estimates for system concepts documented\n•  Initial cost models include high-level process steps and materials \n \nQuestions (MRL Criteria in the form of a question):\n1. Has manufacturing cost estimates for system concepts been developed?    \n2. Have initial cost models been developed which include high-level process steps and materials?\n\nAdditional Considerations: \n• Affordability must be considered even during the early stages of a product life-cycle. Design and process engineers should be aware that designing a product cannot be done in a vacuum.\n• Consider a cost step-down approach, leveraging new/emerging technologies and manufacturing processes, and consider automation.\n• Production cost estimation can be by a number of means including formal Production Cost Models, parametric estimates, production actuals or cost risk bands based on technical maturity.\n• Cost targets can come from should-cost models which use existing internal products, future state products, competitor/best-in-class products, and customer price models.\n• Value stream maps showing high level architecture breakdown with suppliers aligned could be used to show goals for broken out costs.\n• Risks include large cost items, items with large variations, and new items with unknown costs.\n• Cost drivers might be material, labor, overhead, NRE, tooling, yields, rework, repair, etc. for any of the parts of the product life-cycle cost.\n\nLessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• Cost estimates at this stage of the product life-cycle should be considered as ROM estimates and be open to adjustment as the design is firmed up."]
["$F$11",127,"Cost estimates refined based on anticipated production volumes associated with preferred materiel solution. Cost model updated with identified cost drivers (i.e., process variables, manufacturing, materials, and special requirements). Cost driver uncertainty quantified. Cost model supports AoA and ASR. ","Purpose:\nTo break down production cost by identifying drivers in manufacturing, material, and special requirements for alternative designs, and to quantify the uncertainty of cost targets for a list of given assumptions (rate, process, material, production personnel, supplier capability, etc.). To use this information to understand the cost risk in selecting the best alternative. To ensure the validity behind cost targets by reviewing detail process charts and assumptions behind these targets and to quantify the uncertainty of the targets. The program must project production cost by setting cost targets and identifying cost risks to produce the selected alternative.  At the end of MSA, the best available alternatives will be selected to address entrance criteria for TMRR, and you will understand the cost risk this decision presents in achieving program requirements\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments providing a ROM allocation of cost of the production concept.\n\nExamples of ‘Objective Evidence’:\n•  Cost estimates refined based on anticipated production volumes associated with preferred materiel solution and documented\n•  Cost model updated with identified cost drivers (i.e., process variables, manufacturing, materials, and special requirements) and documented\n•  Quantification and documentation of cost driver uncertainty\n•  Analysis of Alternatives (AoA)\n•  Alternative Systems Review (ASR) documentation\n\nQuestions (MRL Criteria in the form of a question):\n1. Have cost estimates been refined based on anticipated production volumes associated with preferred materiel solution?\n2. Has cost model been updated with identified cost drivers (i.e., process variables, manufacturing, materials, and special requirements)?\n3. Has the uncertainty of these cost drivers been quantified?\n4. Does the cost model support AoA and ASR?\n\nAdditional Considerations:  \n•These are budgetary estimates for design trades and should include Basis of Estimate (BOE) considerations.\n•Probable controlling influences on cost include materiel, processes, International Traffic in Arms Regulations (ITAR), capital investment, software, test, personnel, etc.\n•Cost drivers should be considered from all parts of the product life-cycle (e.g., R&D, transition to production, production ramp-up/learning, sustainment, hidden factory, logistics, repair and aftermarket, and disposal costs).\n•Cost targets can come from should-cost models which use similar existing internal products, supply chain competitor products, future state products, customer price models, etc.\n\nLessons Learned: \n•The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle. \n•Quantities and rates at this phase are notional.\n"]
["$G$11",128,"Prototype components produced in a production relevant environment, or simulations drive end-to-end cost models. Cost model includes materials, labor, equipment, tooling/STE/SIE, setup, yield/scrap/rework, WIP, and capability/capacity  constraints.   ","Purpose:  \nTo obtain more fidelity in the production cost estimates by using actual data from building prototypes or by using acceptable simulation models to focus on cost drivers.  At this stage, details in the cost model should address specific manufacturing areas (e.g., materials, labor, equipment, tooling/Special Test Equipment, setup, yield/scrap/rework, Work In Progress, and capability/capacity constraints, etc.) for each prototype being built. The program manager must have confidence in the production cost to adequately assess the affordability of his final decision.  \n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Documented cost model information\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are the end-to-end cost models driven by results of producing (or simulating production of) prototype components in a production relevant environment?\n2. Does the cost model include materials, labor, equipment, tooling, special test equipment, special inspection equipment, setup, yield, scrap, rework, Work in Progress, and capability and/or capacity constraints?\n\n\nAdditional Considerations:  \n•  There should be a reduced reliance on laboratory processes and resources at this stage.\n•  Are design-to-cost (DTC) goals being met? \n•  Is actual cost data from prototype builds being used?\n•  Include quantities in the DTC estimate.  DTC numbers should have more fidelity, if not completely firm, and assumptions should be clearly identified.  \n•  Are cost targets in place and cost drivers clearly understood?\n•  If equipment, tooling, or STE are capitalized by the supplier, it may not be applicable to the analysis. Some level of clarification should be provided on when this data is considered.\n\n\nLessons Learned: \n•  The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n•  Quantities and rates at this phase are still somewhat notional.\n"]
["$H$11",129,"Cost model updated with design requirements, material specifications, tolerances, IMS, results of system/subsystem simulations and  production relevant prototype demonstrations.  ","Purpose:  \nTo obtain more fidelity in the production cost model it should be updated using data from building and testing the prototypes in a relevant environment.  The program needs to have more realism in the production cost by using actual build data or acceptable simulation models to adequately assess affordability of final elected design.  At Milestone B, the best available alternative will be selected to address user requirements, and the cost risk this decision presents will be understood.\n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Documented cost model information\n•  Integrated Master Schedule (IMS)\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Is the cost model up-to-date?\n2.  In addition to prior elements, does the updated cost model include design requirements, material specifications, tolerances, consideration of the Integrated Master Schedule, the results of system and/or subsystem simulations, and demonstrations of production relevant prototypes? \n\n\nAdditional Considerations:  \n•  Include quantities in the Design to Cost (DTC) estimate.  DTC numbers should have more fidelity, if not completely firm, and assumptions should be clearly identified.  \n•  Are cost targets in place and cost drivers clearly understood?\n\n\nLessons Learned: \n•  The cost model needs to be populated actual prototype build data as soon as possible, and it must continually be updated to reflect changes.\n"]
["$I$11",130,"Cost model updated with the results of systems/sub-systems produced in a production representative environment, production plant layout and design, and  obsolescence solutions.","Purpose:  \nTo use demonstrated results from a production representative environment to verify production cost requirements can be met. The program must verify the realism in the cost model by using demonstrated data from a production representative environment to ensure confidence in affordability projections.\n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Cost model information\n•Demonstrated data from a production representative environment\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Is the cost model up-to-date?\n2. In addition to prior elements, does the updated cost model include the results of systems and/or sub-systems produced in a production representative environment, the production plant layout and design, and solutions to obsolescence risks and issues?\n\n\nAdditional Considerations:  \n•Is actual cost data from prototype builds being used?\n•Include updated costs from suppliers including yield/rework which might affect cost of LRIP and FRP manufacturing.\n\n\nLessons Learned: \n•The cost model needs to be populated with actual production representative build data, and it must continually be updated to reflect changes.\n"]
["$J$11",131,"Cost model updated with results of pilot line build.","Purpose:  \nSince MRL 8 occurs in the later stages of the Engineering and Manufacturing Development (EMD) phase within the acquisition cycle, the main focus is to demonstrate the manufacturing process will support program requirements. You should leave this phase with adequate knowledge, obtained through demonstration, that your manufacturing processes are capable of producing affordable systems (i.e. achieving your production target cost requirements).\n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Cost models show the results of the pilot line build\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Is the cost model up-to-date with results of the pilot line build?\n\n\nAdditional Considerations:  \n•None\n\n\nLessons Learned: \n•The key is to continually update your cost models as more data becomes available.\n"]
["$K$11",132,"FRP cost model updated with result of LRIP build. ","Background: \nMRL 9 occurs in the Low-Rate Initial Production (LRIP) phase within the acquisition cycle. The main focus if this phase of the acquisition cycle is to permit an orderly increase in the production rate upon successfully completing operational testing.\n\n\nPurpose:\nMRL 9 occurs in the Low-Rate Initial Production (LRIP) phase within the acquisition cycle. The main focus if this phase of the acquisition cycle is to permit an orderly increase in the production rate upon successfully completing operational testing. The program constantly needs to verify the realism in the cost model by using demonstrated data from a LRIP environment to ensure we have confidence in our affordability projections.  \n\n\nRationale:\nThe program constantly needs to verify the realism in the cost model by using demonstrated data from a LRIP environment to ensure we have confidence in our affordability projections. \n\n\nDefinitions: \n• None\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Cost model information\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has the Full Rate Production (FRP) cost model been updated with the result of the Low \nRate Initial Production (LRIP) build?  \n\n\nAdditional Considerations:  \n•None\n\n\nLessons Learned: \n•The key is to continually update your cost models as more data becomes available.\n"]
["$L$11",133,"Cost model validated against actual FRP cost. ","Background: \nMRL 10 occurs in the Full-Rate Production (FRP) phase within the acquisition cycle. The main focus if this phase of the acquisition cycle is to provide the User quality products on cost and schedule.\n\n\n\nGoal:  \nMake our cost projections.\n\n\n\nRationale:\nThe program demonstrated data supports our affordability projections. \n\n\nDefinitions: \n• None\n\n\nPurpose:  \nMRL 10 occurs in the Full-Rate Production (FRP) phase within the acquisition cycle. The main focus if this phase of the acquisition cycle is to provide the User quality products on cost and schedule. The program demonstrated data supports our affordability projections.\n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n \n\nExamples of ‘Objective Evidence’:\n•Cost model has been validated against actual FRP cost\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has the cost model been validated against actual Full Rate Production (FRP) cost?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \nThe key is to continually update your cost models as more data becomes available."]
["$A$12",134]
["$B$12",135,"C.2 - Cost Analysis"]
["$C$12",136,"Initial manufacturing and quality costs identified.","Purpose:  \nTo begin identifying manufacturing cost implications, and to use this information to understand the risk in pursuing a given approach. To allow decision makers insight into the cost risk and start exploring mitigation options to address identified cost risks.  Developers need to begin the process of identifying manufacturing cost implications for any approach being considered.  \n\n\nSources of Information:  \nThe S&T community, including service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further knowledge. \n\n\nExamples of ‘Objective Evidence’:\n•  Initial manufacturing and quality costs identified and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have initial manufacturing and quality costs been identified?\n\n\nAdditional Considerations:  \n• What are the major impediments to manufacturing with new materials or achieving streamlined manufacturing processes producing more affordable products?\n• What new Warfighter capabilities are being held back due to manufacturing costs or full scale productization solutions?\n\n\nLessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle."]
["$D$12",137,"Potential manufacturing and quality cost drivers and system affordability gaps identified. ","Purpose:  \nTo begin identifying manufacturing cost elements, and to use this information to understand the risk in pursuing a proposed approach. To allow decision makers insight into the cost risk and start exploring mitigation options to address identified cost risks. The S&T community needs to have a basic understanding on the manufacturing cost elements and implications of proceeding with the approach being considered.  \n\nSources of Information:  \nThe S&T community, including service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further knowledge. \n\nExamples of ‘Objective Evidence’:\n•  Potential manufacturing and quality cost drivers and system affordability gaps identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have potential manufacturing and quality cost drivers and system affordability gaps been identified?\n\nAdditional Considerations:  \n• The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms.\n• Things to consider include yields/rates/volumes typical for the product line and/or market for which the technology is intended.\n\nLessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle."]
["$E$12",138,"Analyses conducted to refine manufacturing and quality cost drivers, risks, and development strategy (e.g., lab to pilot to factory). Potential cost reduction and system affordability gap closure strategies identified.","Purpose:  \nTo begin identifying the sensitivity of manufacturing cost drivers, and to use this information to understand the risk in pursuing this approach. To allow decision makers insight into the cost risk and start exploring mitigation options to address identified cost risks. To effectively transition ATD programs one must address manufacturing maturity of the prototypes being developed. Assessing the maturity of the manufacturing processes in ATDs mandates that the S&T and acquisition community work together.  A key output of that assessment is a basic understanding of manufacturing cost drivers and their implications to the production development strategy.  \n\nSources of Information:  \nThe S&T sponsors like the Service S&T organizations and research organizations in academia and industry must begin working with the acquisition community at this time. At the beginning of ATD, overall objectives/goals for cost, schedule and performance that are to be demonstrated should be reviewed with the customer(s) (i.e. Users, S&T, and the Acquisition community).  It is especially critical that the cost goals reflect manufacturing cost considerations and capabilities.\n\nExamples of ‘Objective Evidence’:\n•  Analyses for refining manufacturing and quality cost drivers, risks, and development strategy (e.g., lab to pilot to factory) are documented \n•  Trade-offs in design options based on experiments and initial MOEs are documented\n \nQuestions (MRL Criteria in the form of a question):\n1. Have analyses been conducted to refine manufacturing and quality cost drivers, risks, and development strategy (e.g., lab to pilot to factory)?  \n2. Are trade-offs in design options based on experiments and initial MOEs? \n\nAdditional Considerations:  \n• The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms.\n• Risks include large cost items, items with large variations, and new items with unknown costs.\n• Cost drivers might be material, labor, overhead, NRE, tooling, yields, rework, repair, etc. for any of the parts of the product life-cycle cost.\n\nLessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle."]
["$F$12",139,"Producibility and lifecycle cost risks and issues assessed for preferred materiel solution. Initial cost analysis supports AoA and ASR.","Purpose:  \nTo begin assessing the producibility of the alternatives being considered and ensure production cost models support the AoA.  To use this information to understand the cost risk in selecting the alternatives to proceed with into the TMRR phase. To ensure the validity behind cost targets by reviewing detail process charts and assumptions behind these targets and to quantify the uncertainty of the targets. The program manager must begin projecting production cost and identifying cost risk to produce the alternative being considered.  At the end of this phase the best available alternatives will be selected to proceed into TMRR.  A producibility assessment should be performed to understand the cost risk presented by this decision.\n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Producibility cost risks and issues for the preferred materiel solution assessed and documented \n•  Initial cost analysis support Analysis of Alternatives (AoA) \n•  Initial cost analysis support Alternative Systems Review (ASR) \n\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have producibility and lifecycle cost risks and issues been assessed for the preferred materiel solution?    \n2. Does the initial cost analysis support the Analysis of Alternatives (AoA)?\n3. Does the initial cost analysis support the Alternative Systems Review (ASR)?\n\nAdditional Considerations:  \n•Cost analysis in the AoA can be done with a cost-weighted Pugh Concept Selection (PCS) or Quality Function Deployment (QFD) on those items with costs that can be modeled.\n\nLessons Learned: \n•The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n•To accomplish this, use the data that is generated by a producibility analysis in the AoA and ASR.\n"]
["$G$12",140,"Costs analyzed using prototype component actuals to ensure target costs are achievable.  Decisions regarding design choices, make/buy, capacity, process capability, sources, quality, KCs, yield/rate, and variability influenced by cost models.    ","Purpose:  \nTo obtain more fidelity in the production cost estimates by using data from building prototypes to assess whether target costs are achievable.  To use data generated from cost analyses to make design choices, make/buy, capacity, process capability, sources, quality, key characteristics, yield/rate, and variability assessments. The program manager needs to obtain more realism assessing production cost by using actual prototype build data to ensure that his final decision will result in an affordable product.  At the end of this phase, the best available alternatives will be selected to proceed into EMD. Any cost risk this decision presents in achieving program requirements will be understood.\n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Cost model information reflects prototype cost targets\n•  Cost model influences design choices, make/buy, capacity, process capability, sources, quality, key characteristics, yield/rate, and variability\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have prototype components' actual costs been included in cost analyses to validate achievability of target costs?\n2.  Is the up-to-date cost model being utilized as an input to decisions (trades) on design, make/buy choices, capacity requirements, process capability requirements, sources, quality requirements, Key Characteristics, yield/rate requirements, and variability control requirements?   \n\n   \nAdditional Considerations:  \n•  Are design-to-cost (DTC) goals being met? \n•  Is actual cost data from prototype builds being used?\n•  Include quantities in the DTC estimate.  DTC numbers should have more fidelity, if not completely firm, and assumptions should be clearly identified.  \n•  Are cost targets in place and cost drivers clearly understood?\n•  If equipment, tooling, or STE are capitalized by the supplier, it may not be applicable to the analysis. Some level of clarification should be provided on when this data is considered.\n\nLessons Learned: \n•  The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n•  The need for realism of the cost model requires the use of actual build data as soon as possible.  This data will impact the affordability of alternatives being considered.\n"]
["$H$12",141,"Costs analyzed using prototype system/sub-system actuals to ensure target costs are achievable. Cost targets allocated to subsystems. Cost reduction and avoidance strategies developed. Manufacturing cost drivers for \"Should-Cost\" model provided.","Purpose:  \nTo obtain more fidelity in the production cost model (i.e. at system/sub-system level) by updating it using data from building and testing prototypes in a relevant environment. To use this data to identify cost targets and drivers and begin efforts for cost reduction strategies. To provide this information for the “Should-Cost” models. The program manager needs to obtain more realism assessing production cost by using actual prototype build data to ensure that his final decision will result in an affordable product.  At the end of this phase, the best available alternatives will be selected to proceed into EMD. Any cost risk this decision presents in achieving program requirements will be understood.\n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Cost model information reflects subsystem cost targets\n•  Cost reduction and avoidance strategies\n•  \"Should-Cost\" model\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have prototype system and/or subsystems' actual costs been included in cost analyses to validate achievability of target costs? \n2.  Have cost targets been allocated to subsystems? \n3.  Have cost reduction and avoidance strategies been developed? \n4.  Have manufacturing cost drivers for the \"Should-Cost\" models been provided?\n\n\nAdditional Considerations:  \n•  You need actual build costs to perform an accurate Should-Cost review.\n\n\nLessons Learned: \n•  The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n•  The need for realism of the cost model requires the use of actual build data as soon as possible.  This data will impact the affordability of alternatives being considered.\n"]
["$I$12",142,"Manufacturing costs rolled up to system/sub-system level and tracked against targets.   Detailed trade studies and engineering change requests supported by cost estimates.   Cost reduction and avoidance strategies underway. Manufacturing cost drivers for \"Should-Cost\" model updated.","Purpose:  \nTo use demonstrated results from a production representative environment to verify you can achieve production cost requirements. To use data for trade studies and engineering change and to start implementing your cost reduction strategies. To update the “Should-Cost” model with actuals from the production represented environment.  The program needs to verify the realism of the cost model by using actual cost data from a production representative environment to ensure confidence in system affordability projections. The program needs to use the cost data to make design and manufacturing decisions, and ensure cost reduction activities have started.\n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Cost model information reflects subsystem cost targets\n•Detailed trade studies and engineering change requests supported by cost estimates \n•Cost reduction and avoidance strategies underway\n•Manufacturing cost drivers for \"Should-Cost\" models updated\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have all available actual manufacturing costs been rolled up to the subsystem and/or system level?\n2. Have all available actual manufacturing costs been tracked against targets?\n3. Is the up-to-date cost model and life cycle cost estimate being utilized as an input to decisions on detailed design trade studies and engineering change requests? \n4. Are cost reduction and avoidance strategies being implemented (i.e., underway)?\n5. Have the manufacturing cost drivers for the \"Should-Cost\" models been updated?\n\n\n\nAdditional Considerations:  \n•You need actual build costs to perform an accurate Should-Cost review.\n•Costs for subsystems and materials must be applicable to the objective military system design.\n\n\nLessons Learned: \n•The need for realism of the cost model requires the use of actual build data.  This data will impact the affordability of alternatives being considered.\n•The cost model needs to be populated with actual production representative build data, and it must continually be updated to reflect changes.\n•A program must stay focused on the cost reduction effort at the beginning of EMD, or, when you transition into production, you will use the inefficiencies in the past to justify the cost for production versus what it should cost.\n"]
["$J$12",143,"Costs analyzed using pilot line actuals to ensure target costs are achievable.  Manufacturing cost analysis supports proposed changes to requirements or configuration.  Cost reduction initiatives ongoing. Manufacturing cost drivers for \"Should-Cost\" model updated.","Purpose:  \nTo use demonstrated results from a production representative environment to verify you can achieve production cost requirements. To use data for trade studies and engineering change and to start implementing your cost reduction strategies. To update the “Should-Cost” model with actuals from the production represented environment.  The program needs to verify the realism of the cost model by using actual cost data from a production representative environment to ensure confidence in system affordability projections. The program needs to use the cost data to make design and manufacturing decisions, and ensure cost reduction activities have started.\n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Costs analyzed using pilot line actuals to ensure target costs are achievable\n•Manufacturing cost analysis documented to support proposed changes to requirements or configuration\n•Ongoing cost reduction initiatives documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have pilot line actual costs been included in cost analyses to verify achievability of target costs?\n2. Have proposed changes to requirements or configuration been subject to a manufacturing cost analysis?\n3. Are the cost reduction and cost avoidance initiatives ongoing?\n4. Have the manufacturing cost drivers for the \"Should-Cost\" models been updated?\n\n\n\nAdditional Considerations:  \n•Is actual cost data from pilot line builds being used?\n•Include updated costs from suppliers including yield/rework which might affect cost of LRIP and FRP manufacturing.\n\n\nLessons Learned: \n•Though the cost model may be using actual pilot line build data, it must continually be updated to reflect changes.\n•When updating the cost model with pilot line build cost data, it may be appropriate to leverage similar technology/process data to “tune” the model (e.g., transition from Pilot Line to Production shows standard learning curves may not be appropriate).\n"]
["$K$12",144,"LRIP cost goals met and learning curves analyzed with actual data.  Cost reduction initiatives ongoing. Touch labor efficiency analyzed to meet production rates and elements of inefficiency are identified with plans in place for reduction.","Purpose:  \nTo use demonstrated results from a LRIP line to verify you can achieve your production rate cost requirements. To use the data to analyze the labor efficiency, for Should-Cost estimates, and to identify areas for improvement. The program must constantly verify realism of the cost model by using actual build data from LRIP to ensure confidence in affordability projections. This early production data will set the foundation for future production cost, so you must understand where management emphasis needs to be placed to reduce cost and ensure production cost objectives can be achieved. \n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n \n\nExamples of ‘Objective Evidence’:\n•Documentation that Low Rate Initial Production (LRIP) cost goals been met \n•Learning curve analyzed with actual data \n•Cost reduction initiatives documented \n•Touch labor efficiency analyzed to meet production rates \n•Plans for elements of inefficiency for touch labor available\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have Low Rate Initial Production (LRIP) cost goals been met?\n2. Have the learning curves been analyzed with actual data? \n3. Are the cost reduction and cost avoidance initiatives ongoing? \n4. Has touch labor efficiency been analyzed to meet production rates?\n5. Have elements of inefficiency for touch labor been identified with plans in place for reduction?\n\n\n\nAdditional Considerations:  \n• Is actual cost data from LRIP builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of FRP manufacturing. \n\nLessons Learned: \n• The cost model must be using actual FRP build data, and it should continually be updated as more data becomes available.\n• Use this data to update the Should Cost estimate for the program."]
["$L$12",145,"FRP cost goals met.  Cost reduction initiatives ongoing.","Purpose:  \nMake accurate cost projections. The program actual production build cost data should support affordability projections and provide cost-reduction opportunities. \n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n \n\nExamples of ‘Objective Evidence’:\n•Evidence that FRP cost goals have been met\n•Evidence that cost reduction initiatives are continuing\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have Full Rate Production (FRP) cost goals been met?\n2. Are the cost reduction and cost avoidance initiatives continuing?\n\n\n\nAdditional Considerations:  \n• Is actual cost data from FRP builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of FRP manufacturing. \n\n\nLessons Learned: \n• The cost model should at this point look at obsolescence issues and product refresh. These could change the cost model but might not be considered.\n• If the design changes for any reason the system might be at a lower MRL after the design change. Many organizations will still consider this as Full Rate Production even though process technologies need to change."]
["$A$13",146]
["$B$13",147,"C.3 - Manufacturing Investment Budget"]
["$C$13",148,"Potential manufacturing investment strategy developed.","Purpose:  \nTo begin identify manufacturing cost implications, and to use this information to allow decision makers insight into potential investments to address identified cost risks. Researchers need to begin a process for understanding the manufacturing cost implications of their approaches and to consider investment options to reduce risk.  \n\n\nSources of Information:  \nThe S&T community, including service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further knowledge. \n\n\nExamples of ‘Objective Evidence’:\n•  A potential manufacturing investment strategy developed and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has a potential manufacturing investment strategy been developed? \n\n\nAdditional Considerations:  \n• What are the major impediments to manufacturing with new materials or achieving streamlined manufacturing processes producing more affordable products?\n• What new Warfighter capabilities are being held back due to lack of manufacturing investments or full scale productization solutions?\n\n\nLessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if assessing cost and identifying potential investments to reduce cost starts before hardware development really begins."]
["$D$13",149,"Program/projects have reasonable budget estimates for reaching MRL 3 through experiment. Manufacturing investment budget ROM estimates identified to support industrial base and manufacturing capability gap closure strategies.","Purpose:  \nTo assess whether there are adequate funds budgeted to reach an MRL 3. The S&T community needs to have funds available on R&D programs to address manufacturing maturity, and if it is anticipated that there will be hardware migrating to the next phase, there should be a budget plan to achieve an MRL 3 before completing the R&D effort (i.e., Applied Research or Advanced Technology Development).  \n\nSources of Information:  \nThe S&T community, including service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further knowledge. \n\nExamples of ‘Objective Evidence’:\n•  Program/projects have documented reasonable budget estimates for reaching MRL 3 through experiment \n•  Manufacturing investment budget ROM estimates to support industrial base and manufacturing capability gap closure strategies identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Do the program/projects have reasonable budget estimates for reaching MRL 3 through experiment?\n2. Have manufacturing investment budget ROM estimates been identified to support industrial base and manufacturing capability gap closure strategies?\n\nAdditional Considerations:  \n• The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms and manufacturing investments should be considered to address these issues.\n• Critical issues may involve large cost items, items with large variations, and new items with unknown costs.\n• Cost drivers might be material, labor, overhead, NRE, tooling, yields, rework, repair, etc. for any of the parts of the product life-cycle cost.\n\nLessons Learned: \n• Product affordability has the highest likelihood of being achieved if identification of investments to reduce cost is completed at during hardware development.\n• Budgeting efforts should be instituted to mature the manufacturing processes and to understand the risk."]
["$E$13",150,"Program/projects have reasonable budget estimates for reaching MRL 4 by MS A. Preliminary manufacturing investment budget estimates for manufacturing gap closure recommendations have been developed.","Purpose:  \nTo make sure that there is a reasonable budget to mature the manufacturing process to a MRL 4. To effectively transition S&T or R&D projects one must address the manufacturing maturity of the prototypes being developed. Budgeting the funds to reach a MRL 4 on these projects will require the S&T and acquisition community working together to ensure this is being done.\n\nSources of Information:  \nThe S&T sponsors, like the Service R&D organizations and research organizations in academia and industry, must begin working with the acquisition community at this time. At the beginning of ATD, or when it is known that a project will be transition into acquisition both communities should have overall objectives/goals for cost, schedule and performance to be demonstrated. These objectives should be reviewed with the customers (the users, R&D, and Acquisition communities) on some regular schedule. It is critical that the cost goals reflect manufacturing cost considerations and capabilities.\n\nExamples of ‘Objective Evidence’:\n•  Program/projects have documented reasonable budget estimates for reaching MRL 4 by Milestone (MS) A \n•  Preliminary manufacturing investment budget estimates for manufacturing gap closure recommendations are documented\n \nQuestions (MRL Criteria in the form of a question):\n1. Do the program/projects have reasonable budget estimates for reaching MRL 4 by Milestone (MS) A?\n2. Have preliminary manufacturing investment budget estimates for manufacturing gap closure recommendations been developed? \n\nAdditional Considerations:  \n• The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms and manufacturing investments should be considered to address these issues.\n• Risks include large cost items, items with large variations, and new items with unknown costs.\n• Potential investments might be in material, labor, major NRE (internal and suppliers), tooling, facilities, etc. \n\nLessons Learned: \n• Product affordability has the highest likelihood of being achieved if potential investments to reduce cost are programmed during the process of hardware development.\n• Budgeting efforts should be instituted to mature the manufacturing processes and to understand the risk."]
["$F$13",151,"Manufacturing technology budget initiatives developed and incorporated to reduce costs. Program has reasonable budget estimate for reaching MRL 6 by MS B. Estimate includes capital investment for production relevant equipment. All outstanding MRL 4 risks and issues understood with approved mitigation plans in place. ","Purpose:  \nTo ensure the budget is adequate to achieve MRL 4, and to ensure there are mitigation plans in place to address all outstanding issues from initial Manufacturing Readiness Assessments. The program manager needs to assess the funding available to achieve an MRL 4 by the end of MSA, and if there is not, to identify the risk of proceeding into TMRR.  \n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•   Manufacturing technology initiatives identified and documented \n•   Reasonable manufacturing budget estimate documented to reach MRL 6 by Milestone B\n•   Cost estimates include capital investment for equipment to be used in the production relevant environment\n•   Outstanding MRL 4 risks and issues have approved mitigation plans\n\nQuestions (MRL Criteria in the form of a question):\n1. Have manufacturing technology initiatives been incorporated to reduce costs?\n2. Does the program have a reasonable manufacturing budget estimate to reach MRL 6 by Milestone B?\n3. Does the manufacturing budget estimate include capital investment for equipment to be used in the production relevant environment?\n4. Are all outstanding MRL 4 risks and issues understood with approved mitigation plans in place?\n\nAdditional Considerations:  \n• The government program manager, the contractor, and the service-specific ManTech organization need to firm up what investments need to be made to address affordability on the program.\n• Affordability risks include large cost items, items with large variations, and new items with unknown costs.\n• Potential investments might be in material, labor, major Non Recurring Engineering (NRE) for both Contractor and suppliers, tooling, facilities, etc. \n\nLessons Learned: \n• Product affordability has the highest likelihood of being achieved if investments to reduce cost begin during the process of hardware development.\n• Budgeting efforts should be programmed to mature the manufacturing processes and to understand the risk.\n"]
["$G$13",152,"Program has updated budget estimate for reaching MRL 6 by MS B. All outstanding MRL 5 risks and issues understood with approved mitigation plans in place. ","Purpose:  \nTo ensure the budget is adequate to achieve MRL 6, and to ensure there are mitigation plans in place to address all outstanding issues from initial Manufacturing Readiness Assessments. The program manager needs to assess the funding available to achieve an MRL 6 by the end of TMRR, and if there is not, to identify the risk of proceeding into EMD.  He should build prototypes in a production relevant environment by the end of TMRR.\n\n\nSources of Information:  \nThis data should come from cost estimating, budgeting, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Budget estimate for reaching MRL 6\n•  Approved mitigation plans (MMPs) for MRL 5 risk areas\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Does the program have an updated manufacturing budget estimate for reaching MRL 6 by Milestone B?\n2.  Are all outstanding MRL 5 risks and issues understood with approved mitigation plans in place?\n\n\nAdditional Considerations:  \n•  Does equipment, tooling, or STE at the supplier require investment? If so, who will fund it?\n•  Consider leveraging outside investments (e.g., SBIR, IRAD, ManTech, etc.).\n\n\nLessons Learned: \n•  Product affordability has the highest likelihood of being achieved if investments to reduce cost have begun by this stage.\n"]
["$H$13",153,"Program has reasonable budget estimate for reaching MRL 8 by MS C. Estimate includes capital investment for production-representative equipment by CDR and pilot line equipment by MS C. All outstanding MRL 6 risks and issues understood with approved mitigation plans in place. ","Purpose:  \nTo assess the EMD budget to determine if it is reasonable for maturing the manufacturing processes to an MRL 8, and that all manufacturing risk items in TMRR have appropriate mitigation plans in place. The program manager needs understand what budget is necessary to achieve an MRL 8 at the end of EMD and that there are approved mitigation plans in place to address the risk identified in TMRR before proceeding into EMD.  One of the major requirements for EMD will be to build product in Pilot Line environment, and this requires discussion between the customer and supplier to fully understand funds needed to achieve this. If this is not addressed before entering EMD, it could become a major problem during EMD.\n\n\nSources of Information:  \nThis data should come from cost estimating, budgeting, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Budget estimate for reaching MRL 8\n•  Cost estimate includes capital investment\n•  Approved mitigation plans (MMPs) for MRL 6 risk areas\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Does the program have a reasonable manufacturing budget estimate for reaching MRL 8 by Milestone C?\n2.  Does the manufacturing budget estimate include capital investment for equipment to be used on the pilot line prior to Milestone C? \n3.  Are all outstanding MRL 6 risks and issues understood with approved mitigation plans in place?\n\n\nAdditional Considerations:  \n•  Does equipment, tooling, or STE at the supplier require investment? If necessary, has this been considered in overall program cost estimates?\n\n\nLessons Learned: \n•  Product affordability has the highest likelihood of being achieved if investments to reduce cost have begun before EMD.\n"]
["$I$13",154,"Program has updated budget estimate for reaching MRL 8 by MS C. All outstanding MRL 7 risks and issues understood with approved mitigation plans in place. ","Purpose:  \nTo use demonstrated results from a production representative environment to ensure production requirements can be met. To use data for updating budgets to achieve an MRL 8 and ensure all manufacturing risks that are identified have approved mitigation plans in place. The program must use actual product build data from a production representative environment to demonstrate confidence in the budget to achieve program maturity requirements. The program needs to use this data to update budgets as required.\n\n\nSources of Information:  \nThis data should come from cost estimating, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Updated budget estimate for reaching MRL 8\n•Approved mitigation plans (MMPs) for MRL 7 risk areas\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Does the program have an updated manufacturing budget estimate for reaching MRL 8 by Milestone C? \n2. Are all outstanding MRL 7 risks and issues understood with approved mitigation plans in place?\n\n\n\nAdditional Considerations:  \n•You need actual build costs compared to original estimates to validate whether return on investment (ROI) is adequate to justify further investment.\n•Costs for subsystems and materials must be applicable to the objective military system design.\n\n\nLessons Learned: \n•The need for realism of the cost model requires the use of actual build data.  This data will impact the affordability of alternatives being considered.\n•The cost model needs to be populated with actual production representative build data, and it must continually be updated to reflect changes.\n•A program must stay focused on the cost reduction effort at the beginning of EMD, or, when you transition into production, you will use the inefficiencies in the past to justify the cost for production versus what it should cost.\n"]
["$J$13",155,"Program has reasonable budget estimate for reaching MRL 9 by the FRP decision point. Estimate includes investment for LRIP and FRP. All outstanding MRL 8 risks and issues understood with approved mitigation plans in place. ","Purpose:  \nTo use demonstrated data from a pilot line environment to verify production requirements are being met and that all manufacturing risk mitigation plans are working. The program must ensure confidence in the budget by using actual hardware build data from a pilot line environment.\n\n\nSources of Information:  \nThis data should come from cost estimating, budgeting, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Budget estimate for reaching MRL 9\n•Cost estimate includes investment for Low Rate Initial Production (LRIP) and Full Rate Production (FRP)\n•Approved mitigation plans (MMPs) for MRL 8 risk areas\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Does the program have a reasonable manufacturing budget estimate for reaching MRL 9 by the Full Rate Production (FRP) decision point?\n2. Does the manufacturing budget estimate include investments for Low Rate Initial Production (LRIP) and Full Rate Production (FRP)?\n3. Are all outstanding MRL 8 risks and issues understood with approved mitigation plans in place?\n\n\n\nAdditional Considerations:  \n•Do actual build costs validate whether return on investment (ROI) is adequate to justify further investment?\n•Is the customer manufacturing representative involved and agree to the MRL assessment and the Manufacturing Maturation Plan(s) to mitigate risks?\n\n\nLessons Learned: \n•Though the budget may be relying on actual pilot line build data, it must continually be updated to reflect changes. The key is to use the data to make informed program investments.\n"]
["$K$13",156,"Materials controlled to specifications in LRIP.   Materials proven and validated as adequate to support FRP.","Purpose:  \nTo use demonstrated results from a LRIP line to verify an adequate budget exists to achieve production rate requirements and that all manufacturing risks areas have approved mitigation plans. The program manager must verify the realism of the budget by using actual build data from a LRIP environment to ensure confidence in meeting production requirements. This early production data will set the foundation for future production efforts and management needs to understand where emphasis must be placed to reduce cost and ensure production objectives can be achieved. \n\n\nSources of Information:  \nThis data should come from cost estimating, budgeting, design and manufacturing departments.\n \n\nExamples of ‘Objective Evidence’:\n•Budget estimate for reaching MRL 9\n•Approved mitigation plans (MMPs) for MRL 9 risk areas\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Does the program have a reasonable manufacturing budget estimate for Full Rate Production (FRP)?\n2. Are all outstanding MRL 9 risks and issues understood with approved mitigation plans in place?\n\n\n\nAdditional Considerations:  \n• Is actual cost data from LRIP builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of FRP manufacturing. \n\n\nLessons Learned: \n• You must continually update budgets as more data becomes available and use this data to ensure program objectives can be met."]
["$L$13",157,"Production budgets sufficient for production at required rates and schedule to support funded program. ","Purpose:  \nTo verify you have an adequate budget to achieve you production rate requirements. The program manager must continually verify the realism of the budget by using actual build data from a production environment to ensure confidence in meeting production requirements. \n\n\nSources of Information:  \nThis data should come from cost estimating, budgeting, design and manufacturing departments.\n \n\nExamples of ‘Objective Evidence’:\n•Production budget sufficient for production at the required rates and schedule\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Is the production budget sufficient for production at the required rates and schedule to support the funded program?\n\n\nAdditional Considerations:  \n• Is actual cost data from FRP builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of FRP manufacturing. \n\n\nLessons Learned: \n• You must continually update budgets as more data becomes available and use this data to ensure program objectives can be met."]
["$A$14",158,"D - Materials (Raw Materials, Components, Sub-assemblies and Sub-systems)","Materials: Requires an analysis of the risks associated with materials (including basic/raw materials, components, semi-finished parts, and subassemblies).\n\n\"Materials\" include analysis of the maturity, availability, supply chain management, and special handling (i.e. government furnished property, shelf life, security, hazardous materials, storage environment, etc.)."]
["$B$14",159,"D.1 - Maturity"]
["$C$14",160,"New material properties and characteristics surveyed and identified for research (e.g., manufacturability, quality).","Purpose:  \nTo begin to identify material properties identified for research, and use this information to allow decision makers insight into potential investments to address those properties. Researchers must begin understanding and developing material properties needed to provide future capability to the war fighter.  \n\n\nSources of Information:  \nS&T organizations such as service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. Professional organizations provide the ideal forum for exchange of new ideas and approaches to exploring the boundaries of discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further the state of knowledge. \n\n\nExamples of ‘Objective Evidence’:\n•  New material properties and characteristics for research (e.g., manufacturability, quality) surveyed, identified and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have new material properties and characteristics been surveyed and identified for research (e.g., manufacturability, quality)?\n\n\nAdditional Considerations:  \n• Consider historically known material limitations and compatibility issues. \n\n\nLessons Learned: \n•  Historically materials manufacturability and producibility issues are ignored early in R&D so identifying potential materials manufacturability issues at this point will reduce effort in later stages of the product life cycle."]
["$D$14",161,"Potential effects of new material properties on design application manufacturability and quality predicted based on research.","Purpose:  \nTo begin predicting material properties and characteristics from research being performed. S&T organizations such as service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. If it is anticipated that there will be hardware migrating to the next phase there should be some effort to predict the material properties and characteristics anticipated before completing the R&D effort.\n\nSources of Information:  \nS&T organizations such as service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. Professional organizations provide the ideal forum for exchange of new ideas and approaches to exploring the boundaries of discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further the state of knowledge.\n\nExamples of ‘Objective Evidence’:\n•  Based on research, potential effects of new material properties on design application manufacturability and quality have been predicted and documented\n \nQuestions (MRL Criteria in the form of a question):\n1. Have the potential effects of new material properties on design application manufacturability and quality been predicted based on research?\n\nAdditional Considerations:  \n• Consider historically known material limitations and compatibility issues. \n• Will new materials or product development concepts most likely be required?\n• Is a new technology or process available to process the material?\n\nLessons Learned: \n• Historically materials manufacturability and producibility issues are ignored early in R&D. \n• This is a good time to identify materials producibility and manufacturability risks to the R&D community before transitioning so they can understand potential risk of making the product."]
["$E$14",162,"Effects of new material properties on design concept manufacturability and quality validated using experiments and models. ","Purpose:  \nTo assess the properties of the materials being proposed to transition into acquisition. To effectively transition R&D projects one must address the manufacturing maturity of the materials being developed. \n\nSources of Information:  \nS&T organizations such as service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. At the beginning of technology development, or when it is known that a project will be transition into acquisition both communities should have an overall objectives/goals for materials characteristics to be demonstrated. These objectives should be reviewed with the customers (the users, R&D, and Acquisition communities) on some regular schedule.\n\nExamples of ‘Objective Evidence’:\n•  The effects of new material properties on design concept manufacturability and quality using experiments and models are documented\n \nQuestions (MRL Criteria in the form of a question):\n1. Have the effects of new material properties on design concept manufacturability and quality been validated using experiments and models?\n\nAdditional Considerations:  \n• Consider historically known material limitations and compatibility issues. \n• Consider manufacturing issues in life-cycle cost, quality, and cycle time from literature, simulation or technology demonstration.\n\nLessons Learned: \n• Materials producibility and manufacturability risks must be presented to the to the R&D community before transitioning so they can understand potential risk of making the product."]
["$F$14",163,"New materials and components for preferred materiel solution demonstrated in a laboratory environment.","Purpose:  \nTo assess the maturity of proposed materials to understand manufacturing risk to program.  The materials produced in a laboratory environment should be assessed to determine additional steps needed to achieve program requirements.\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\nExamples of ‘Objective Evidence’:\n•  Documentation showing that projected materials demonstrated in a laboratory environment \n•  Material traceability documentation\n\nQuestions (MRL Criteria in the form of a question):\n1. Have new materials and components for the preferred materiel solution been demonstrated in a laboratory environment?\n\nAdditional Considerations:  \n• Consider materials use in similar products or technology demonstrators and lessons learned from each.\n• New materials maybe new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n\nLessons Learned: \n• Materials producibility and manufacturability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives.\n"]
["$G$14",164,"Materials manufactured or produced in a prototype environment (may be in a similar application/program).  Maturation efforts in place to address new material production risks for technology demonstration.","Purpose:  \nTo understand how the materials are being produced for prototypes and compare those results to what is required to achieve program objectives.  To do initial assessment of materials in a production relevant environment. Use this information to put into place the appropriate mitigation plans to address all outstanding issues/risks for maturity your materials. The program manager must assess material maturity in the earliest acquisition phases.\n\n\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Documentation showing that materials have been manufactured or produced in a prototype environment (e.g., production relevant)\n•  Maturation efforts documented for new material production risks and issues\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have all materials been manufactured or produced in a prototype environment (e.g., production relevant)?\n2.  Were any materials manufactured or produced a similar application and/or program?\n3.  Are material maturation efforts in place to address risks and issues from producing new materials for the technology demonstration?\n\n\nAdditional Considerations:  \n•  Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n•  Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n\n\nLessons Learned: \n•  Materials producibility and manufacturability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives.\n•  Develop a plan to address material maturity risk as the program goes through TMRR and transitions into EMD.\n"]
["$H$14",165,"Material maturity verified through technology demonstration articles.  Preliminary material specifications in place. Material properties adequately characterized.","Purpose:  \nTo assess the material maturity demonstrated in the TMRR phase versus what is required to achieve program objectives. To have adequate material specifications in place to transition to EMD and ensure that the material properties have been adequately assessed with appropriate mitigation plans in place to proceed to EMD. The program manager must understand the material maturity achieved from producing the prototype units in TMRR and the risk of proceeding to EMD. \n\n\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Material verification documentation\n•  Material specifications\n•  Material properties characterization documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have technology demonstration articles (e.g., prototypes) verified maturity of materials?\n2.  Are preliminary material specifications in place?\n3.  Have material properties been adequately characterized to enable producing prototypes in a production relevant environment?\n\n\nAdditional Considerations:  \n•  Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n•  Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n\n\nLessons Learned: \n•  Execute the plan to address material maturity risk as the program goes through TMRR and transitions into EMD.\n"]
["$I$14",166,"Material maturity sufficient for pilot line build. Material specifications approved.","Purpose:  \nTo use demonstrated results from a production representative environment to verify material specifications are sufficiently mature to achieve production requirements.  \n\n\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Material maturity sufficient for pilot line build\n•Material specifications have been approved\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Is the material maturity sufficient for a pilot line build?\n2. Have material specifications been approved?\n\n\nAdditional Considerations:  \n•Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n\n\nLessons Learned: \n•Material maturity must be continually assessed in this program phase, and the focus should be on transition to production.\n"]
["$J$14",167,"Materials proven and validated during EMD as adequate to support LRIP.  Material specifications stable. ","Purpose:  \nTo use demonstrated data from a pilot line environment to verify material specifications are mature and sufficient to achieve production requirements for LRIP.  \n\n\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Material verification proven and validated during Engineering and Manufacturing Development (EMD) as adequate to support Low Rate Initial Production (LRIP)\n•Material specifications stable\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have the materials been proven and validated during Engineering and Manufacturing Development (EMD) as adequate to support Low Rate Initial Production (LRIP)?\n2. Are material specifications stable (i.e., no changes to specifications)?\n\n\n\nAdditional Considerations:  \n•Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n\n\nLessons Learned: \n•Material maturity must be continually assessed in this program phase, and the focus will be on transition to production (i.e. LRIP).\n"]
["$K$14",168,"Material is controlled to specification in LRIP.   Materials proven and validated as adequate to support FRP.","Purpose:  \nTo assure all material is being controlled by a specification in LRIP. To use the demonstrated results from a LRIP line to verify adequate maturity to achieve production rate requirements and that all manufacturing risk areas have approved mitigation plans.  Materials should have achieved maturity by this phase.\n\n\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Material verification documentation\n•Material specifications controlled\n•Material proven and validated as adequate to support FRP\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are all materials meeting specifications duringLow Rate Initial Production (LRIP)? \n2. Have the materials been proven and validated as adequate to support FRP?\n\n\n\nAdditional Considerations:  \n• Are new materials being introduced at this stage of the program? If so, you need to carefully assess whether potential impacts to production might outweigh the apparent benefit of any new materials.\n\n\nLessons Learned: \n• Maturity of material specifications must be continually evaluated. This area historically represents a significant amount of risk and must be closely monitored."]
["$L$14",169,"Materials controlled to specifications in FRP. ","Purpose:  \nTo verify the material specifications are under control as the program achieves production rate requirements.  The program must constantly assess material maturity in achieving production requirements.\n\n\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Material specifications\n•Material specifications controlled\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are all materials meeting specifications during Full Rate Production (FRP)?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• It is crucial to continually assess material maturity data to ensure program objectives will be met."]
["$A$15",170]
["$B$15",171,"D.2 - Availability"]
["$C$15",172,"Global trends for material availability, obsolescence, and DMSMS surveyed and identified for research.","Purpose:  \nTo begin to identify material properties identified for research, and use this information to allow decision makers insight into availability of materials. Researchers must begin understanding and developing material properties needed to provide future capability to the war fighter.  \n\n\nSources of Information:  \nS&T organizations such as service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. Professional organizations provide the ideal forum for exchange of new ideas and approaches to exploring the boundaries of discovery and invention and availability.\n\n\nExamples of ‘Objective Evidence’:\n•  Global trends for material availability, obsolescence, and DMSMS surveyed and identified for research and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have global trends for material availability, obsolescence, and DMSMS been surveyed and identified for research?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$15",173,"Material availability, obsolescence, and DMSMS gaps identified.","Purpose:  \nTo begin assessing the availability of the materials required. The S&T community needs to begin assessing the availability of the material for their programs and to start addressing any issues identified.  If it is anticipated that there will be hardware migrating to the next phase there should be some effort to predict the material availability issues before completing the R&D effort.\n\nSources of Information:  \nS&T organizations such as service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. Professional organizations provide the ideal forum for exchange of new ideas and approaches to exploring the boundaries of discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further the state of knowledge.\n\nExamples of ‘Objective Evidence’:\n•  Material availability, obsolescence, and DMSMS gaps identified and documented\n \nQuestions (MRL Criteria in the form of a question):\n1. Has material availability, obsolescence, and DMSMS gaps been identified?\n\nAdditional Considerations:  \n• Consider historically known material limitations and compatibility issues. \n• Will new materials or product development concepts most likely be required?\n• Is a new technology or process available to process the material?\n\nLessons Learned: \n• Historically materials manufacturability and producibility issues are ignored early in R&D. \n• This is a good time to identify materials producibility and manufacturability risks to the R&D community before transitioning so they can understand potential risk of making the product."]
["$E$15",174,"Material availability, obsolescence, and DMSMS gap closure strategy defined.","Purpose:  \nTo assess the material availability of projects being proposed to transition into acquisition.  To effectively transition S&T projects one must address the availability issues of the materials being developed for R&D projects being considered for acquisition. \n\nSources of Information:  \nThe S&T sponsors like the Service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. At the beginning of R&D or when it is known that a project may transition into acquisition, both communities should have overall objectives/goals for cost, schedule and performance to be demonstrated. These objectives should be reviewed with the customers (i.e., users, S&T, and Acquisition community) on some regular schedule. It is critical that the cost goals reflect manufacturing cost considerations and capabilities.\n\nExamples of ‘Objective Evidence’:\n•  Material availability, obsolescence, and DMSMS gap closure strategy is defined and documented\n \nQuestions (MRL Criteria in the form of a question):\n1. Has material availability, obsolescence, and DMSMS gap closure strategy been defined?\n\nAdditional Considerations:  \n• Consider historically known material limitations and compatibility issues. \n• Will new materials or product development concepts most likely be required?\n• Is a new technology or process available to process the material?\n\nLessons Learned: \n• Historically materials manufacturability and producibility issues are ignored early in R&D where we start transitioning hardware from the labs to acquisition.   \n• This is a good time to identify materials producibility and manufacturability risks to the R&D community before transitioning so they can understand potential risk of making the product and assessing the material availability. "]
["$F$15",175,"Projected lead times identified for all difficult to obtain, difficult to process, or hazardous materials. Quantities and lead times estimated. Material availability risks and issues for preferred materiel solution considered in AoA. Mitigation plans incorporated in SEP for the preferred system concept.","Purpose:  \nTo assess the availability of proposed materials to understand the manufacturing risk to the program.  The lead times for proposed materials for the program should be identified at this stage. Those with long lead times need to be closely monitored as the program transitions into acquisition.\n\n\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\nExamples of ‘Objective Evidence’:\n•  Projected lead times identified and documented for all difficult to obtain, difficult to process, or hazardous materials \n•  Material quantities and lead times estimated and documented\n•  Analysis of Alternatives (AoA)\n•  SEP includes material mitigation plans\n\nQuestions (MRL Criteria in the form of a question):\n1. Have projected lead times been identified for all difficult-to-obtain, difficult-to-process, or hazardous materials?\n2.  Have material quantities and lead times been estimated?\n3. Have material availability risks and issues for the preferred materiel solution been considered in AoA?\n4. Have mitigation plans been incorporated in SEP for the preferred system concept?\n\nAdditional Considerations:  \n• Consider historically known material limitations and compatibility issues. \n• Will new materials or product development concepts most likely be required?\n• Is a new technology or process available to process the material?\n\nLessons Learned: \n• Hardware availability must be closely monitored as the program transitions from the R&D phase to Acquisition, and it is important to understand and assess lead times of critical materials needed to support the TMRR phase.\n"]
["$G$15",176,"Availability risks and issues addressed for prototype build. Significant material risks identified for all materials. Planning initiated to address scale-up issues.  ","Purpose:  \nTo determine if material will be available to produce the prototype units. To use this information to put into place appropriate mitigation plans to address all outstanding issues/risks for materials availability.\n\n\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Material availability issues for prototype build documented\n•  Significant material risks documented\n•  Plans addressing scale-up issues available\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have material availability risks and issues been addressed for the prototype build? \n2.  Have all materials been analyzed for availability risks and issues?\n3.  Are there any significant material availability risks?\n4.  Has planning been initiated to address availability for scale-up?\n\n\nAdditional Considerations:  \n•  Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n•  Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n•  Consider issues associated with shipment and control of Communications Security (COMSEC) units (can significantly affect schedule).\n\n\nLessons Learned: \n•  Materials availability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives.\n•  Develop a plan to address material availability risk as the program goes through TMRR and transitions into EMD.\n"]
["$H$15",177,"Availability risks and issues addressed to meet EMD build. Long-lead items identified. Potential obsolescence issues identified. Components assessed for future DMSMS risk. ","Purpose:  \nTo assess the material availability for the EMD phase and understand the risk. To begin addressing potential DMSMS (Diminishing Manufacturing Sources and Materials Shortages) and obsolescence risk of the materials being proposed. The program manager must understand the availability of the material required to produce the EMD units and the associated risk. Addressing DMSMS issues should begin as early as possible in the design process where there are options to address these issues.\n\n\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Material availability issues documentation\n•  Long-lead items list\n•  Obsolescence/DMSMS plans/documentation\n\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have material availability risks and issues been addressed to meet Engineering and Manufacturing Development (EMD) build?\n2.   Have materials, items, components, etc. been identified which require a long lead time to obtain?\n3.  Have materials, items, components, etc. been identified which are at risk of obsolescence during the product life cycle?\n4.  Have materials, items, components, etc. been identified which are at risk from Diminishing Manufacturing Sources and Material Shortages during the product life cycle? \n\n\nAdditional Considerations:  \n•  Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n•  Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n•  Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n•  Has the program office planned and budgeted for DMSMS?\n\n\nLessons Learned: \n•  Materials availability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives.\n•  Develop a plan to address material availability risk as the program goes through TMRR and transitions into EMD. \n•  Addressing DMSMS issues must begin as early as possible. Often a program has not budgeted to address DMSMS.\n•  DMSMS is difficult to address, and to do it adequately, it must be done in the early stages of the process before the design matures.\n"]
["$I$15",178,"Availability risks and issues addressed to meet LRIP builds. Long lead procurements identified and mitigated. Obsolescence plan in place. DMSMS mitigation strategies for components in place.","Purpose:  \nTo look forward to see if there are any issues for material availability for LRIP. To start putting into place strategies to minimize obsolescence/DMSMS issues. The program needs to consistently monitor material availability and work obsolescence/DMSMS issues.\n\n\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Long lead procurements identified/planned for Low Rate Initial Production (LRIP)\n•Obsolescence/DMSMS plans/documentation in place\n•Mitigation strategies in place for DMSMS components\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have material availability risks and issues been addressed to meet Low Rate Initial Production (LRIP) builds?\n2. Have materials, items, components, etc. which require a long lead time been identified and mitigated for Low Rate Initial Production (LRIP)?\n3.  Is a plan in place for materials, items, components, etc. that are identified as at risk of obsolescence during the product life cycle?\n4. Are mitigation strategies in place for materials, items, components, etc. that are identified as at risk from Diminishing Manufacturing Sources and Material Shortages during the product life cycle?\n\n\nAdditional Considerations:  \n• Identification of long lead items may be too late if you are within the lead-time for these long-lead items for scheduled LRIP production.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n• Has the program office planned and budgeted for DMSMS?\n\n\n\nLessons Learned: \n•The program manager must continually assess material availability and manage DMSMS issues as the program moves into production.\n"]
["$J$15",179,"Availability risks and issues managed for LRIP. Long lead procurement initiated for LRIP. Availability issues addressed to meet FRP builds.","Purpose:  \nTo assess and address the material availability for LRIP and begin reviewing material issues for FRP.\n\n\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Long lead procurements initiated for Low Rate Initial Production (LRIP) - Purchase Orders or Contracts in place\n•Material availability risk issues minimized for Low Rate Initial Production (LRIP)\n•Availability issues addressed to meet FRP builds - Obsolescence/DMSMS plans/documentation\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are all materials availability risks and issues being managed (i.e., mitigations plans being executed) for Low Rate Initial Production (LRIP)?\n2. Have procurements for all long lead materials been initiated for initial production (LRIP)?\n3. Are materials availability risks and issues being addressed (i.e., mitigation plans) to meet FRP builds?\n\n\nAdditional Considerations:  \n• Initiation of long lead items may be much too late if you are within the lead-time for these long-lead items for scheduled LRIP production.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n• Has the program office planned and budgeted for DMSMS?\n\nLessons Learned: \n•The program manager must continually assess material availability and manage DMSMS issues as the program moves into production (LRIP and FRP).\n"]
["$K$15",180,"Long lead procurement initiated for FRP. Availability risks and issues managed for FRP. ","Purpose:  \nTo assess the material availability for FRP and ensure there are no significant risks.\n\n\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Long lead procurements initiated for FRP\n•Availability risk issues identified for FRP\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have long lead procurements been initiated for Full Rate Production (FRP)? \n2. Are all materials availability risks and issues being managed (i.e., mitigations plans being executed) for Low Rate Initial Production (LRIP)? \n\nAdditional Considerations:  \n• Initiation of long lead items may be much too late if you are within the lead-time for these long-lead items for scheduled FRP production.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n\nLessons Learned: \n• The program manager must continually assess material availability and manage DMSMS issues as the program moves into FRP."]
["$L$15",181,"All material availability risks and issues managed. ","Purpose:  \nTo assess the material availability for FRP and make sure there are no significant risks.\n\n\nSources of Information:  \nThis data should come from material, supplier management, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Material availability risk issues resolved and documented\n•Material availability issues documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are all materials availability risks and issues being managed for FRP?\n\n\nAdditional Considerations:  \n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n\n\nLessons Learned: \n• The program manager must continually assess material availability and manage DMSMS issues as the program is in FRP. "]
["$A$16",182]
["$B$16",183,"D.3 - Supply Chain Management"]
["$C$16",184,"Global trends for supply chain capability and capacity surveyed. ","Purpose:  \nTo begin predicting material properties and characteristics from research being performed. S&T organizations such as service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. If it is anticipated that there will be hardware migrating to the next phase there should be some effort to predict the supply chain capability and capacity gap closure strategy before completing the R&D effort.\n\n\nSources of Information:  \nS&T organizations such as service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. Professional organizations provide the ideal forum for exchange of new ideas and approaches to exploring the boundaries of discovery and invention.  Supply chain capability and capacity surveys are necessary. \n\n\nExamples of ‘Objective Evidence’:\n•  Global trends for supply chain capability and capacity surveyed and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have global trends for supply chain capability and capacity been surveyed?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$16",185,"Potential supply chain capability and capacity gaps identified.","Purpose:  \nTo begin predicting material properties and characteristics from research being performed. S&T organizations such as service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. If it is anticipated that there will be hardware migrating to the next phase there should be some effort to predict the supply chain capability and capacity gap closure strategy before completing the R&D effort.\n\nSources of Information:  \nS&T organizations such as service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. Professional organizations provide the ideal forum for exchange of new ideas and approaches to exploring the boundaries of discovery and invention.  Supply chain capability and capacity research is necessary. \n\nExamples of ‘Objective Evidence’:\n•  Potential supply chain capability and capacity gaps been identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have potential supply chain capability and capacity gaps been identified?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$E$16",186,"Supply chain capability and capacity gap closure strategies defined.","Purpose: \nTo evaluate whether or not there are potential suppliers that are capable of addressing the required technology as well as material solutions of the customer requirements. Since 70 to 80 percent of the cost and risk of any weapon system is in the supply chain, an assessment of the potential supply chain capability must be conducted during this phase. An assessment of supply chain capability identifies the needs of the program versus the established potential suppliers.  Contractors need to establish very early if they need to build time into the schedule to develop a particular skill set, processes, or material in-house or within the supply base to meet program requirements.\n\nSources of Information: \nThe Quality Manager, Supply Chain Manager, Design Engineer, Manufacturing Engineer.\n\nExamples of ‘Objective Evidence’:\n•  Supply chain capability and capacity gap closure strategies are defined and documented\n \nQuestions (MRL Criteria in the form of a question):\n1. Have supply chain capability and capacity gap closure strategies been defined?\n\nAdditional Considerations:\n• Are there new manufacturing processes needed to produce the design?\n• Have lists been developed to identify new equipment, tooling or test equipment that may be needed to produce the design?\n• Have the make/buy decisions been initiated?\n• Can the industrial base support Technology Development (TD) concepts?\n• Have potential supplier survey topics been identified? \n\nLessons Learned: \n• Gain a macro level view of the financial viability of the supply base.  Just because a supplier is technically able to perform the work does not mean that they will be around long enough to provide the product if their financial position is not sound. \n• Attend as many design reviews as possible to gain an understanding of the product and its unique features.  Make a list of these features/design concerns, and have the prime contractor address each concern to understand if the design is producible."]
["$F$16",187,"Survey of potential supply chain sources for preferred materiel solution completed. Supply chain capability and capacity analyses considered in the AoA.","Purpose: \nTo do an assessment for potential supply chain sources is in effect a manufacturing feasibility evaluation to determine: (1) Identification of manufacturing technologies and processes needed; (2) Materials/processes needed for producibility; (3) Potential tradeoff in design options and supply chain ability to support these options; and (4) Supply chain industrial base capabilities as well as potential gaps/risks. During this phase of the acquisition cycle it is important to assess the supply chain industrial base capabilities by completing a supply chain survey to identify gaps/ risks for the preferred concepts and key technology components as well as potential key processes. A survey of supply chain capability indicates that the prime contractor has done the necessary research to understand the potential suppliers for this particular product/program and and/or identified the gaps that exist and mitigation plans to close those gaps.\n\n\nSources of Information: \nThe Quality Manager, Supply Chain Manager, Production Manager, Product Engineer.\n\nExamples of ‘Objective Evidence’:\n•  Potential supply chain sources survey documented for the preferred materiel solution\n•  Analysis of Alternatives (AoA)\n\nQuestions (MRL Criteria in the form of a question):\n1. Has a survey for potential supply chain sources for the preferred materiel solution been completed?\n2. Have supply chain capability and capacity analyses been considered in the AoA?\n\nAdditional Considerations:\n•  Is the prime satisfied that the proposed design can be manufactured, assembled, tested, packaged, and delivered in sufficient quantity, at an acceptable cost to the customer on schedule with the identified potential supply base?\n•  Has the prime established criteria for selection and evaluation of its suppliers?\n•  Has the prime conducted an on-site assessment of potential suppliers?  When?\n•  Critical/new suppliers may need to be tracked more carefully due to potential program risk.\n\nLessons Learned: \n•  Open communication between the government, prime contractor and the supply base throughout the program execution is critical to success.\n•  Review and consider past performance data on supplier capabilities and consider “similar to” supply chains\n•  One needs to discuss and understand where the engineering and technical support is located for the manufacturing facility and how that support communicates to the manufacturing location.\n"]
["$G$16",188,"Potential supply chain sources identified and evaluated as able to support prototype build.","Purpose: \nTo ensure suppliers can produce prototype components in a production relevant environment.  To identify potential supply chain sources and evaluate their capability to support prototype build. To initiate an industrial base capability assessment to identify potential supply chain manufacturing sources (i.e. sole/single/foreign source suppliers and suppliers of technologies with potential obsolescence issues). Although it is not expected that suppliers would have a complete factory and supply chain established this early in a program, it is necessary to obtain some baseline knowledge on critical manufacturing processes, potential production scale up efforts, and potential supply chain issues. The bottom line is potential supply chain sources must be identified and capable to support prototype build by either having or planning to obtain the ability, technical expertise, capacity, willingness, manpower and equipment to fully support build of the designed product.\n\n\nSources of Information: \nThe majority of the information regarding prototype build in a production relevant environment will come from the manufacturing/industrial engineering and quality departments.\n \n\nExamples of ‘Objective Evidence’:\n•Documentation showing supply chain sources identified and evaluated as able to support prototype build\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have potential supply chain sources been identified and evaluated as able to support prototype build?\n\n\nAdditional Considerations:\n•Does a formal and effective sub-tier supplier management process exist?\n•Are there preliminary assessments within the supply chain highlighting any concerns relative to tooling and/or testing that may impact prototype build?\n\n\nLessons Learned: \n•One of the important steps for achieving this requirement is to work with the contractor/ supplier on what represents a prototype production for your programs. \n•Understanding the long lead material, equipment, and resource situation are key to understanding where risk potentially lies in the prototype plan.  An understanding of these elements will help direct focus in the appropriate direction to provide for a successful prototype program. \n•Some type of formal communication of expectations and requirements that must be flowed down to the supply chain.  There must also be a channel in which the supply chain can flow up questions and concerns.\n"]
["$H$16",189,"Lifecycle Supply Chain requirements updated.  Critical suppliers list updated.  Supply chain plans in place (e.g. teaming agreements, etc.) supporting an EMD contract award.","Purpose: \nTo develop: (1) An updated critical supplier list; (2) Life cycle supply chain requirement (i.e. single source/ sole source/ foreign sources, suppliers with long lead items, suppliers with potential DMSMS issues) and (3) Updated supply chain plans (e.g. teaming agreements, Corporate Leverage Contracting (CLC), etc.) in place leading to an EMD contract award.  During the TMRR phase an examination of the producibility of a proposed design is needed to allow for trade analysis decisions to be made on critical suppliers’ cost, performance and schedule. It is significantly easier to make changes during this phase where changes potentially have greater impact on key performance metrics and life cycle cost. Given that 70 to 80 percent of the Bill of Material (BOM) is in the supply chain, suppliers contribute immensely in this area. Lifecycle supply chain requirements refer to all phases of the intended acquisition category being understood and relayed to the supply chain.  \n\n\nSources of Information: \nMost information can be obtained from supplier management, logistics, engineering, production, and quality departments. \n \n\nExamples of ‘Objective Evidence’:\n•  Program supply chain list\n•  Supply chain policies/procedures and contracts/Purchase Orders (POs)\n•  Critical suppliers list\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have requirements for the supply chain throughout the product lifecycle been updated?  \n2.  Has the critical suppliers list been updated?\n3.  Are plans in place that include a supply chain with sufficient resources, capabilities, and capacities to support an awarding of an engineering and manufacturing (EMD) contract award?\n\n\nAdditional Considerations:\n•  Is there capacity at each of the identified suppliers to take on this workload?\n•  Is there an understanding and ranking of the most “critical” suppliers, for quality and delivery?\n•  Is there a list of supplier capability/capacity requirements and goals?\n•  Have Critical/new supplier been identified for risk documentation?\n\n\nLessons Learned: \n•  Understand why a certain supplier is chosen for a particular part.  Ask if there has been similar product produced at that supplier and received by the consuming plant.  Ask about the working relationships between the parties.\n•  Critical suppliers should understand the program schedule and be able to talk to any areas that may be a problem.  \n•  Understand the supplier selection criteria as well as their past performance if available.\n"]
["$I$16",190,"Effective supply chain management processes defined, documented, and in place. Plan developed for predictive indicators. Assessment of critical first tier supply chain completed (i.e., capability, capacity, etc.). ","Purpose: \nFor the prime to maintain: (1) Well defined and documented supply chain management processes and procedures; (2) An initial plan to develop a set of supplier predictive indicators; and (3) Assessment results from first tier critical suppliers (e.g. capability, capacity, etc.). Management of the supply chain has become critical to achieving program cost, schedule, and performance objectives. The percentage of work performed at the suppliers and the risk they present to the program is significant, and to adequately manage a program requires insight into how suppliers are performing and ensuring they are adequately managed.  \n\n\nSources of Information: \nMost information can be obtained from supplier management, industrial engineering, production, and quality departments.  \n\n\nExamples of ‘Objective Evidence’:\n•Supply chain management process defined, documented and in place\n•Management of suppliers using metrics for tracking and managing supplier performance\n•Plan developed for establishing predictive indicators\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Is there an effective supply chain management process defined, documented and in place?\n2. Has a plan been developed for use of predictive indicators for risks in the supply chain?\n3. Has an adequate assessment of the critical first tier supply chain been completed (i.e., capability, capacity, etc.)?\n\n\n\nAdditional Considerations:\n•Has an adequate assessment of the first tier supply chain been completed?\n•Is management aware of supplier performance metrics and ready to take action if needed?\n•How are “high risk” suppliers managed? Is it different than less risky suppliers?\n•Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n•Does the contractor have an adequate process in place to identify and ensure program requirements are properly flowed down to supplier/subcontractors?\n•Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n•Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n•What criteria are used to determine the capability/capacity of subcontractors and critical vendors/suppliers?\n•Identify the major/critical subcontractors/vendors for this program.\n•Are subcontractor/vendor delivery schedules compatible with production schedules?\n•How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?\n\n\nLessons Learned: \n•The problems in supplier management are the same in dealing with the prime: Not identifying and flowing down the right requirements, not selecting the best supplier to effectively achieve your requirements, supplier performance not adequately monitored to spot problems early, supplier requirements are not adequately integrated into an overall integrated program schedule.\n•Review tracking of supplier performance metrics by senior management. Is the tracking reviewed regularly at the appropriate management level? Have any non-conformances or late deliveries been followed up on at the root cause/ corrective action level?\n•The key to proper assessment of suppliers is not the set of audit questions; it is the conduct of the audit itself and the follow up actions.\n•Review risk mitigation plans. Are the mitigation actions sufficient to mitigate documented risks? Is there a feedback loop to ensure mitigation actions are completed?\n•High-risk suppliers need to be managed differently than less risky suppliers.\n"]
["$J$16",191,"Assessment of critical second and lower tier supply chain completed.  Robust requirements flow down processes in place and verified. Supplier compliance with program requirements and changes validated. Plan for predictive indicators for use in production updated. Supply chain adequate to support LRIP.","Purpose: \nTo ensure: (1) Supply chain adequacy to support LRIP; (2) Adequate assessment of critical second and lower tier supply chain capability; (3) That proper processes and discipline are in place to ensure effective supplier requirements are being flowed down. Most prime contractors rely on their first tier suppliers to manage the lower tier. This methodology would be adequate if the prime could be certain that the first tier and second tier critical suppliers have the proper discipline to manage lower tier sub-contractors. Some primes perform Process Validation Audits (PVAs) on their first tier critical suppliers to ensure proper sub-tier management. Assessment of critical second and lower tier supply chain evidence assures the customer that the contractor has performed a thorough review of the critical manufacturing processes needed to produce the intended design at the supplier.  \n\n\nSources of Information: \nThe majority of the information needed will come from the manufacturing/industrial engineering, supplier management and quality departments.  \n\n\nExamples of ‘Objective Evidence’:\n•Program supply chain documented and adequate to support LRIP\n•Assessment of the critical second and lower tier supply chain completed \n•A robust and disciplined requirements flow-down process verified and in place \n•Plan for predictive indicators updated for use in production \n•Supplier compliance with program requirements and changes validated \n•Supply chain policies/procedures and contracts/Purchase Orders (POs)\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has an assessment, including capability and capacity, of second and lower tier suppliers whose performance could seriously jeopardize the successful achievement of a program’s cost, schedule, technical, or supportability requirements been completed?\n2. Are robust flow-down processes for requirements in place?\n3. Have the requirements flow-down processes been verified?\n4. Has compliance with flowed-down requirements and any flowed-down changes been validated throughout the supply chain?\n5. Has the plan been updated for use of predictive indicators during production?\n6. Does the supply chain management system indicate that the supply chain is adequate, sufficient, and ready to support initial production (LRIP)?\n\n\n\nAdditional Considerations:\n•Has an adequate assessment of the first tier supply chain been completed?\n•Is management aware of supplier performance metrics and ready to take action if needed?\n•How are “high risk” suppliers managed? Is it different than less risky suppliers?\n•Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n•Does the contractor have an adequate process in place to identify and ensure program requirements are properly flowed down to supplier/subcontractors?\n•Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n•Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n•What criteria are used to determine the capability/capacity of subcontractors and critical vendors/suppliers?\n•Identify the major/critical subcontractors/vendors for this program.\n•Are subcontractor/vendor delivery schedules compatible with production schedules?\n•How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?\n\n\nLessons Learned: \n•Prime contractor, generally, has limited controls over other company divisions \n•Look for a feedback loop to indicate that flow-down requirements are understood and verified. It is NOT sufficient for prime to communicate requirements without confirming understanding and acceptance.\n•Implementing a proactive and robust requirements flow-down process is a challenge.  An undefined change control process is another area that can be a source of risk for the program. \n•Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area.  \n•Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality.  Suppliers may not understand the application of their products into the overall system.  The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II.\n"]
["$K$16",192,"Long term agreements in place where practical. Prime's supplier management metrics (including thresholds and goals) in place and used to manage risks. Predictive indicators to manage suppliers in place. Supply chain is stable and adequate to support FRP.","Purpose: \nTo ensure: (1) Supply chain adequacy to support LRIP; (2) Adequate assessment of critical second and lower tier supply chain capability; (3) That proper processes and discipline are in place to ensure effective supplier requirements flow down. Most prime contractors rely on first tier suppliers to manage the lower tier. This methodology would be adequate if the prime is certain that the first tier and second tier critical suppliers have the proper discipline to manage lower tier sub-contractors.  Some primes perform Process Validation Audits (PVAs) on first tier critical suppliers to ensure proper sub-tier management. Assessment of critical second and lower tier supply chain evidence assures the customer that the contractor has performed a thorough review of critical supplier manufacturing processes to produce the intended design.  \n\n\nSources of Information: \nThe majority of the information needed will come from the manufacturing/industrial engineering, supplier management and quality departments. \n\n\nExamples of ‘Objective Evidence’:\n•Program supply chain stability\n•Supply chain adequate to support FRP\n•Prime's supplier management metrics (including thresholds and goals) in place \n•Predictive indicators in place to manage suppliers\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are long term supplier agreements in place where practical?\n2. Are the supply chain management metrics, which include thresholds and goals in place?\n3. Are the supply chain management metrics utilized to manage risks?\n4. Is the use of predictive indicators to identify risks in the supply chain in place?\n5. Does the supply chain management system indicate that the supply chain is adequate, sufficient, and ready to support rate production (FRP)?\n\n\n\nAdditional Considerations:\n• Is management aware of supplier performance metrics and ready to take action if needed?\n• Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n• Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n• Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n• Are subcontractor/vendor delivery schedules compatible with production schedules?\n• How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?\n\n\nLessons Learned: \n• Look for a feedback loop to indicate that flow-down requirements are understood and verified. It is NOT sufficient for prime to communicate requirements without confirming understanding and acceptance.\n• Implementing a proactive and robust requirements flow down process is a challenge.  An undefined change control process is another area that can be a source of risk for the program. \n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area.  \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality.  Suppliers may not understand the application of their products into the overall system.  The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
["$L$16",193,"Supply chain proven and supports FRP requirements.","Purpose: \nTo demonstrate that the prime contractor has all the required elements of Supply Chain Management in place, and is actively managing the supply chain. Program requirements have been communicated and confirmed and supplier quality and delivery performance tracking metrics are in place. To the extent possible, long term agreements with critical suppliers are in place and are managed effectively.  At this phase of the acquisition, the program has entered Full Rate Production (FRP).  FRP entrance criteria have been accepted and passed. As many suppliers as is prudent should be signed to long-term agreements to provide price stability and availability of required materials. The Prime should have metrics in place to monitor the performance of suppliers along with systematic plan for dealing with underperforming suppliers. The supply base should have predictive indicators in place to call attention to potential problems in a proactive manner. The entire supply base should be known and stable with all suppliers aware of and able to support FRP quantities and delivery dates. Planning must be in place to improve the successful move to FRP and eliminate disruptions.  \n\n\nSources of Information: \nThe majority of the information needed will come from the manufacturing/industrial engineering, supplier management and quality departments. \n\n\nExamples of ‘Objective Evidence’:\n•Program supply chain list\n•Supply chain stable with appropriate contracts/Purchase Orders (POs)\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has the supply chain been proven in production and supports Full Rate Production (FRP)?\n\n\nAdditional Considerations:\n• Are long term agreements in place where practical?\n• Are the Prime's supplier management metrics (including thresholds and goals) in place and being used to manage risk?\n• How does the contractor control their suppliers' suppliers (i.e., lower tier suppliers)?  How does the contractor evaluate their suppliers' subcontract management program?\n• Are predictive indicators in place to manage suppliers?\n• Does evidence of communications with suppliers and validation of requirements exist?\n• Does the contractor understand and maintain performance data of sub-tiers to quantify and manage risk associated with incoming material?\n\n\nLessons Learned: \n• It is critical to look for objective, not anecdotal, evidence to illustrate supplier compliance. Review the metrics for supplier management and demonstrate progression. \n• Ask to see evidence of sub-tier performance data and the problem solving process validation through corrective action reports from the sub-tier suppliers.  \n• Capacity plans that address full rate production should focus on potential supplier bottlenecks. \n• Ask prime to show you how predictive indicators have helped them in proactively avoiding supplier problems before they become a disaster. \n• Ask when and how often supplier metrics are presented to senior leadership and look for examples of how suppliers are informed of their lackluster performance.\n• Risk reduction measures need to be taken for those sub-tier suppliers that have shown a history of poor performance to the defined metrics.  "]
["$A$17",194]
["$B$17",195,"D.4 - Special Handling (i.e., GFP, shelf life, security, HAZMAT, storage environment, etc.) "]
["$C$17",196,"Hazardous materials identified and safety procedures in place. ","Purpose:\nTo develop a basic understanding of what materials will be required for manufacturing and to identify material handling risk to support the analysis of alternatives in the Applied Research phase. This product life-cycle level includes a basic identification of the manufacturing concepts and materials through hypotheses.  \n\n\nSources of Information:\nThe majority of information at this level will come from paper studies and hypotheses.\n\n\nExamples of ‘Objective Evidence’:\n•  Hazardous materials identified and safety procedures are in place and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have hazardous materials been identified and safety procedures are in place?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$17",197,"Initial evaluation of potential regulatory requirements and special handling concerns completed.\nRaw materials and components assessed for special handling and potential regulatory requirements.","Purpose:\nTo develop a basic understanding of what materials will be required for manufacturing and to identify material handling risk to support the analysis of alternatives in the MSA phase.\nThis product life-cycle level includes a basic identification of the manufacturing concepts and materials through paper studies.  \n\nSources of Information:\nThe majority of information at this level will come from paper studies and analyses.\n\nExamples of ‘Objective Evidence’:\n•  Raw materials and components for special handling and potential regulatory requirements assessed and documented\n \nQuestions (MRL Criteria in the form of a question):\n1. Have raw materials and components been assessed for special handling and potential regulatory requirements?\n\nAdditional Considerations:\n• Is there a potential impact due to Environmental Safety and Occupational Health (ESOH) related laws (Clean Air Act, Clean Water Act, Toxic Substances Control Act, National Environmental Policy Act, System Safety, Hazardous Materials Management, etc.)?\n\nLessons Learned:\n• Special handling and ESOH considerations will impact all programs as they move forward. \n• Multiple laws will impact the program.\n• Multiple guides and instructions are available for program use as the program moves forward."]
["$E$17",198,"ESH compliance risk identified. List of hazardous materials identified and alternatives evaluated. Special handling procedures applied in the lab. Special handling concerns assessed. ","Purpose: \nTo develop an understanding of manufacturing feasibility and risk to support the analysis of alternatives in the MSA phase. In this stage of the product life-cycle level validation of manufacturing concepts begins through analytical or laboratory experiments.  Materials and/or processes have been characterized for environmental concerns but further evaluation and demonstration is required.  Experimental hardware models have been developed in a laboratory environment that may possess limited functionality.\n\nSources of Information: \nThe majority of information for this sub-thread will come from paper studies and analysis or from early laboratory experiments.\n\nExamples of ‘Objective Evidence’:\n•  Environmental, Safety, and Health (ESH) compliance risk identified\n•  List of hazardous materials identified and alternatives evaluated \n•  Special handling procedures applied and documented in lab environment\n•  Special handling concerns been assessed and documented\n \nQuestions (MRL Criteria in the form of a question):\n1. Has ESH compliance risk been identified?\n2. Has a list of hazardous materials been identified and alternatives evaluated?\n3. Have special handling procedures been applied in the lab?\n4. Have special handling concerns been assessed?\n\nAdditional Considerations:\n• Is there a potential impact due to Environmental Safety and Occupational Health related laws (Clean Air Act, Clean Water Act, Toxic Substances Control Act, National Environmental Policy Act, System Safety, Hazardous Materials Management, etc.)?\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n\nLessons Learned: \n• Special handling and ESOH considerations will impact all programs as they move forward. \n• Multiple laws will impact the program.\n• Multiple guides and instructions are available for program use as the program moves forward.\n• At this point you probably have actual materials in your hands, if you were not impacted before you may be now.  \n• Know the laws, DOD policies and regulations related to the handling of hazardous materials.  There are several requirements of program offices, make sure that these requirements are flowed down to the contractor so that you will be able to do your job."]
["$F$17",199,"ESH compliance risk mitigated in lab environment. List of hazardous materials updated and alternatives assessed. Special handling procedures applied and disposal procedures evaluated. Special handling requirements identified, analyzed, and documented in the SEP. ","Purpose: \nTo identify several potential material solutions and to assess the costs and risks associated with the preferred material approach.  The exit criteria for this phase should include a Preliminary Hazards List (PHL) and Environmental, Safety, and Health (ESH) criteria.  Special handling requirements for hazardous materials and processes should have been identified. Special handling procedures applied in the lab.  ESH considerations for the next phase should have been identified to include initial ESH planning. This phase looks at the capability to build the item in a laboratory environment.  This includes technologies and manufacturing processes that might have ESH considerations to include cost considerations.  At this point you should have determined the following: (1) Mfg. Feasibility determined (can you build it?); (2) Mfg. Processes Identified (focus on processes with ESH considerations); (3) Key Processes Identified and alternatives identified for those nasty processes; (4) Producibility Assessments Initiated\n\n\nSources of Information: \nThe primary source of information should be the AoA study (report).\n\n\nExamples of ‘Objective Evidence’:\n•  Environmental, Safety, and Health (ESH) compliance risk documented\n•  List of hazardous materials updated and documented \n•  Special handling requirements identified and documented \n•  Special handling procedures applied and documented in lab environment\n•  Material list with Safety Data Sheets\n•  Security classification guides documenting appropriate packaging, escorting, storage and movement procedures\n•  Documented SEP\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has Environmental, Safety, and Health (ESH) compliance risks been mitigated in the lab environment?\n2. Has a list of hazardous materials been updated?\n3. Has the updated list of hazardous materials been assessed for alternative materials?\n4. Have special handling procedures been applied in a laboratory environment? \n5. Have disposal procedures been evaluated?\n6. Have special handling requirements been identified, analyzed, and documented in the SEP?\n\n\nAdditional Considerations:\n• Does the program have access to an ESH professional?\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n\n\nLessons Learned: \n• Special handling and ESH considerations will impact all programs as they move forward. \n• Multiple laws will impact the program (Clean Air Act, Clean Water Act, Toxic Substances Control Act, National Environmental Policy Act, System Safety, Hazardous Materials Management, etc.).\n• Multiple guides and instructions are available for program use as the program moves forward.\n• From here on you may be working with hazardous materials, but the best option is to design hazardous materials out of your program.  Know the laws, DOD policies and regulations related to the handling of HAZMAT.  There are several requirements of program offices; make sure that these requirements are flowed down to the contractor so that you will be able to do your job.\n"]
["$G$17",200,"ESH requirements and special handling procedures applied in production relevant environment. Special handling requirement gaps identified. New special handling processes demonstrated in lab environment. ","Purpose: \nTo design and fabricate at the component level and determine cost, schedule and performance suitability.  A secondary goal is not to have to address Environment, Safety, and Health (ESH) or special handling considerations.  The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal.  In the early stages of technology development effort the role of manufacturing is to “influence the design and plan for production.”  The following are some design and planning concerns: (1) Environment, Safety, and Health (ESH); (2) Programmatic Environmental, Safety, and Health Evaluation (PESHE); (3) National Environmental Policy Act (NEPA) Planning; (4) Human Systems Integration; (5) Systems Safety (Explosives, Laser, Noise, ManPrint, etc.); (6) Hazardous Material Management (HMM); (7) Pollution Prevention Planning.\n\n\nSources of Information: \nThe majority of information for assessing risk should come from contractor design and build documentation. \n\n\nExamples of ‘Objective Evidence’:\n•  Special handling requirement gaps documented \n•  Documentation for special handling procedures applied in a production relevant environment\n•  Documentation for new special handling processes has been demonstrated in lab environment\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have Environmental, Safety, and Health (ESH) requirements been applied in the production relevant environment?\n2.  Have special handling procedures been applied in the production relevant environment? \n3.  Have special handling requirement gaps been identified?\n4.  Have new special handling processes been demonstrated in lab environment? \n\n\nAdditional Considerations:\n•  Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n\n\nLessons Learned: \n•  NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program.  Ensure time is scheduled for these activities.  \n•  You are most likely going to be working with HAZMAT.  Plan for your involvement, but the best option is to design hazardous materials out of your program.  Know the laws, DOD policies and regulations related to the handling of hazardous materials.  There are several requirements of program offices; make sure that these requirements are flowed down to the contractor so that you will be able to do your job.\n"]
["$H$17",201,"ESH requirements addressed and documented. Special handling procedures demonstrated in production relevant environment. Plans to address special handling requirement gaps, risks, and issues complete. Manufacturing assessed for material storage and waste handling risks.","Purpose: \nTo design, fabricate, assemble and integrate at the sub-system level and determine cost, schedule and performance suitability.  A secondary goal for this sub-thread is not to have to address Environment, Safety, and Health (ESH) or special handling considerations.  The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal. In the early stages of technology development effort the role of manufacturing is to “influence the design and plan for production.”  The following are some design and planning concerns: (1) Environment, Safety, and Health (ESH); (2) Programmatic Environmental, Safety, and Health Evaluation (PESHE); (3) National Environmental Policy Act (NEPA) Planning; (4) Human Systems Integration; (5) Systems Safety (Explosives, Laser, Noise, ManPrint, etc.); (6) Hazardous Material Management (HMM); (7) Pollution Prevention Planning.\n\n\nSources of Information: \nThe majority of information for assessing risk should come from contractor design and build documentation.  \n\n\nExamples of ‘Objective Evidence’:\n•  Special handling procedures documented – Technical Data, Work Instructions, etc.\n•  Special handling requirement procedures plan/documentation\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have ESH requirements been addressed and documented?\n2.  Have special handling procedures been demonstrated in a production relevant environment?\n3.  Are the plans to address the special handling requirement gaps, risks, and issues complete?\n4.  Has manufacturing been assessed for material storage and waste handling risks?\n\n\nAdditional Considerations:\n•  Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n\n\nLessons Learned: \n•  NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program.  Ensure time is scheduled for these activities.\n•  You are most likely going to be working with HAZMAT.  Plan for your involvement, but the best option is to design hazardous materials out of your program.  Know the laws, DOD policies and regulations related to the handling of hazardous materials.  There are several requirements of program offices; make sure that these requirements are flowed down to the contractor so that you will be able to do your job.\n"]
["$I$17",202,"ESH compliance demonstrated in production representative environment. Special handling procedures applied in production representative environment. Special handling procedures developed and annotated on work instructions for pilot line. Hazardous material storage and disposal plan in place for the pilot line.","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability.  A secondary goal for this sub-thread is not to have to address ESH or special handling considerations.  The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal. In the early stages of technology development effort the role of manufacturing is to “influence the design and plan for production.”  The following are some design and planning concerns: (1) Environment, Safety, and Health (ESH); (2) Programmatic Environmental, Safety, and Health Evaluation (PESHE); (3) National Environmental Policy Act (NEPA) Planning; (4) Human Systems Integration; (5) Systems Safety (Explosives, Laser, Noise, ManPrint, etc.); (6) Hazardous Material Management (HMM); (7) Pollution Prevention Planning.\n\n\nSources of Information: \nThe majority of information for assessing risk should come from contractor design and build documentation.  \n\n\nExamples of ‘Objective Evidence’:\n•Special handling procedures applied in a production representative environment \n•Special handling procedures developed and annotated on work instructions\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has ESH compliance been demonstrated in a production representative environment?\n2. Have special handling procedures been applied in a production representative environment?\n3. Have special handling procedures been developed and annotated on the work instructions to be used for the pilot line?\n4. Is there a hazardous material storage and disposal plan in place for the pilot line?\n\n\n\nAdditional Considerations:\n•Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n\n\nLessons Learned: \n•NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program.  Ensure time is scheduled for these activities.\n•You are most likely going to be working with HAZMAT. Plan for your involvement, but the best option is to design hazardous materials out of your program.  Know the laws, DOD policies and regulations related to the handling of hazardous materials.  There are several requirements of program offices, make sure that these requirements are flowed down to the contractor so that you will be able to do your job.\n•Designing out hazardous material or material that needs special attention pays off.  It costs eighty dollars or more to handle, treat and dispose of hazardous material for every one dollar spent to purchase the material!\n"]
["$J$17",203,"ESH compliance demonstrated in pilot line production. Special handling procedures applied in pilot line environment and demonstrated in EMD or technology insertion programs.  Special handling risks and issues pose managed for LRIP. All work instructions contain special handling provisions as required. Hazardous material storage and disposal plan evaluated and in place for LRIP.","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability.  A secondary goal for this sub-thread is not to have to address ESH or special handling considerations.  The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal.\n\n\nSources of Information: \nThe majority of information for assessing risk should come from contractor design and build documentation.  \n\n\nExamples of ‘Objective Evidence’:\n•Special handling procedures documented and applied in a pilot line environment – Technical Data, Work Instructions, etc.\n•Special handling procedures demonstrated in Engineering and Manufacturing Development (EMD) or technology insertion programs\n•Special handling risk issues documented and minimized for Low Rate Initial Production (LRIP)\n•All work instructions contain special handling provisions as required\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has ESH compliance been demonstrated during pilot line production?\n2. Have special handling procedures been applied in a pilot line environment?  \n3. Have special handling procedures been demonstrated in Engineering and Manufacturing Development (EMD) or technology insertion programs?    \n4. Have the special handling risks and issues been managed for Low Rate Initial Production (LRIP)?\n5. Do all work instructions contain required special handling provisions?\n6. Have hazardous material storage and disposal plans for LRIP been evaluated?\n7. Are hazardous material storage and disposal plans in place for LRIP?\n\n\n\nAdditional Considerations:\n•Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n\n\nLessons Learned: \n•NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program.  Ensure time is scheduled for these activities.\n•You are most likely going to be working with HAZMAT.  Plan for your involvement, the best option is to design hazardous materials out of your program.  Know the laws, DOD policies and regulations related to the handling of hazardous materials.  There are several requirements of program offices, make sure that these requirements are flowed down to the contractor so that you will be able to do your job.\n•Designing out hazardous material or material that needs special attention pays off.  It costs eighty dollars or more to handle, treat and dispose of hazardous material for every one dollar spent to purchase the material!\n"]
["$K$17",204,"ESH compliance demonstrated in LRIP. Special handling, and hazardous material storage and disposal procedures demonstrated in LRIP environment.  Special handling, and hazardous material storage and disposal risks and issues managed for FRP. ","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability.  A secondary goal for this sub-thread is not to have to address ESH or special handling considerations.  The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal.  \n\n\nSources of Information: \nThe majority of information for assessing risk should come from contractor design and build documentation.  \n\n\nExamples of ‘Objective Evidence’:\n•Special handling procedures documented – Technical Data, Work Instructions, etc.\n•Special handling requirement procedures plan/documentation\n•Special handling risk issues identified and minimized\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has ESH compliance been demonstrated in LRIP?\n2. Have special handling procedures been demonstrated in a Low Rate Initial Production (LRIP) environment? \n3. Have hazardous material storage and disposal procedures been demonstrated during Low Rate Initial Production (LRIP)?\n4. Are risks and issues from special handling and hazardous materials storage and disposal managed for Full Rate Production (FRP)?\n\n\n\nAdditional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n\n\nLessons Learned: \n• Monitor compliance."]
["$L$17",205,"ESH compliance demonstrated in FRP. Special handling and hazardous material storage and disposal procedures effectively implemented in FRP.   ","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability.  A secondary goal for this sub-thread is not to have to address ESH or special handling considerations.  The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal.  \n\n\nSources of Information: \nThe majority of information for assessing risk should come from contractor design and build documentation.  \n\n\nExamples of ‘Objective Evidence’:\n•Special handling procedures documented – Technical Data, Work Instructions, etc.\n•Special handling requirement procedures plan/documentation\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has ESH compliance been demonstrated in FRP?\n2. Have the special handling procedures been effectively implemented in FRP?  \n3. Have the hazardous material storage and disposal procedures been effectively implemented in FRP?  \n\n\n\nAdditional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n\n\nLessons Learned: \n• Continue to monitor risks and continuously improve."]
["$A$18",206,"E - -Process Capability & Control","Process Capability and Control: Requires an analysis of the risks that the manufacturing processes are able to reflect the design intent (repeatability and affordability) of key characteristics.\n\nProcess Capability and Control includes modeling and simulation (product and process), manufacturing process maturity, and process yields and rates."]
["$B$18",207,"E.1 - Modeling & Simulation (Product & Process)"]
["$C$18",208,"Modeling and simulation approaches/tools identified to support manufacturing and quality activities.","Purpose:  \nTo begin addressing the need for models to simulate the manufacturing of the projects and to use this information to understand the risk in pursuing this approach. To allow decision makers insight into the risk and start exploring mitigation options to address those risks identified. Modeling and simulations in these early phases can provide needed in-sight into the risk of the products being considered and developed.\n\n\nSources of Information:  \nThe S&T community, including service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention. Modeling and simulation approaches and tools applied.\n\n\nExamples of ‘Objective Evidence’:\n•  Modeling and simulation approaches/tools identified to support manufacturing and quality activities and documented \n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have modeling and simulation approaches/tools been identified to support manufacturing and quality activities?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$18",209,"Modeling and simulation in development initiated.","Purpose:  \nTo begin addressing the need for models to simulate the manufacturing of the projects and to use this information to understand the risk in pursuing this approach. To allow decision makers insight into the risk and start exploring mitigation options to address those risks identified. Modeling and simulations in these early phases can provide needed in-sight into the risk of the products being considered and developed.\n\nSources of Information:  \nThe S&T community, including service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further knowledge. \n \nExamples of ‘Objective Evidence’:\n•  Modeling and simulation in development initiated and documented \n\nQuestions (MRL Criteria in the form of a question):\n1. Have modeling and simulation in development been initiated?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address what models and simulation tools we will use to assess the manufacturing operation earlier in the acquisition process."]
["$E$18",210,"Manufacturing and quality gaps for system concepts identified using modeling and simulation.","Purpose:  \nTo begin identifying manufacturing risk for the alternative being considered, and to use this information to understand the manufacturing risk to help manage the transition. To see some high level process charts to support this effort and for new processes have some data available based on a minimum of experimental data.  The project needs identify manufacturing risk to produce each alternative being considered.  At the end of this Pre-MSA phase the best alternatives will be selected to address user requirements to proceed into MSA.  \n\nSources of Information:  \nThe S&T sponsors like the Service S&T organizations and research organizations in academia and industry must begin working with the acquisition community at this time. At the beginning of ATD, overall objectives/goals for cost, schedule and performance should be reviewed with the customer (i.e., users, R&D organizations, and the Acquisition community).\n\nExamples of ‘Objective Evidence’:\n•  Manufacturing and quality gaps for system concepts have been identified and documented using modeling and simulation \n\nQuestions (MRL Criteria in the form of a question):\n1. Have manufacturing and quality gaps for system concepts been identified using modeling and simulation?\n\nAdditional Considerations:  \n• Do process flow charts accurately capture all critical process operations?\n\nLessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address what models and simulation tools we will use to assess the manufacturing operation. \n• To aid in this assessment, the program should address the manufacturing approach being considered using as a minimum a high-level process flow chart and identify the risk associated with using this approach. "]
["$F$18",211,"Production modeling and simulation tools utilized to define manufacturing and quality requirements for preferred materiel solution. Modeling and simulation results considered in the AoA.","Purpose:  \nTo understand the modeling/simulation approach that will be used to identify the risk drivers in manufacturing for alternatives being considered, and to use this information to understand the insight in the risk in selecting the best alternative.  The program needs to project and understand the tools used to assess manufacturing risk to produce proposed alternatives.  At the end of this phase the best alternatives will be selected to address user requirements to proceed into TMRR.\n\n\nSources of Information:  \nThis data should come from model and simulation, design and manufacturing departments.\n\nExamples of ‘Objective Evidence’:\n•  Production modeling and simulation approaches for process or product identified and documented for the preferred materiel solution\n•  Analysis of Alternatives (AoA)\n\nQuestions (MRL Criteria in the form of a question):\n1. Have production modeling and simulation tools been utilized to define manufacturing and quality requirements for the preferred materiel solution?\n2. Have modeling and simulation results been considered in the AoA?\n\nAdditional Considerations:  \n• Do process flow charts accurately capture all critical process operations?\n\nLessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must start addressing what models and simulation tools we will use to assess the manufacturing operation earlier in the acquisition process. \n• Value Stream Maps are a useful tool to understand manufacturing constraints.\n"]
["$G$18",212,"Initial modeling & simulations (product or process) developed at the component level and used to determine constraints.","Purpose:  \nTo obtain more fidelity in assessing manufacturing risk by using data from prototypes builds or acceptable simulation models to focus on risk drivers. The program needs to obtain more realism in assessing manufacturing risk/constraints by using real data or acceptable simulation models.  This will ensure adequate assessment of manufacturing risk posed by the final design.  At the end of this phase the best alternative will be selected to proceed into EMD.\n\n\nSources of Information:  \nThis data should come from modeling and simulation, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Simulation models developed at the component level\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have the initial modeling & simulation (product or process) been developed at the component level and used to determine constraints?\n\n\nAdditional Considerations:  \n•  Do process flow charts accurately capture all critical process operations?\n•  The models/simulations at this stage should be at the component level and address specific manufacturing areas to identify constraints for each prototype being built (e.g., materials, labor, equipment, tooling/Special Test Equipment (STE), setup, yield/scrap/ rework, Work In Progress (WIP), and capability/capacity). \n•  Are supplier processes included in the overall flow, including lead times for critical items?\n\n\nLessons Learned: \n•  Value Stream Maps are a useful tool to understand manufacturing constraints.\n•  The simulation model needs to incorporate actual build data as soon as possible.\n"]
["$H$18",213,"Initial modeling & simulations developed at the sub-system or system level, and used to determine system constraints.","Purpose:  \nTo obtain more fidelity in the models/simulations at the sub-system or system level by updating previous component models using data from building and testing prototypes in a relevant environment.  The program must obtain more realism in assessing manufacturing risk/constraints by using actual build data or highly realistic simulation models.  This will ensure an adequate assessment of manufacturing risk posed by the final design.  At the end of this phase the best alternative will be selected to proceed into EMD.\n\n\nSources of Information:  \nThis data should come from model/simulation, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Simulation models developed at the technology sub-system or system level\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have the initial modeling & simulation been developed at the sub-system or system level?\n2.  Have the initial modeling & simulations been used to determine system constraints?\n\n\nAdditional Considerations:  \n•  The models/simulations at this stage should be at the subsystem level and address specific manufacturing areas to identify constraints for each prototype being built (e.g., materials, labor, equipment, tooling/Special Test Equipment (STE), setup, yield/scrap/ rework, Work In Progress (WIP), and capability/capacity). \n•  Are supplier processes included in the overall flow, including lead times for critical items?\n\n\nLessons Learned: \n•  Any simulation models need to incorporate actual build data where available, continually be updated to reflect changes, and be representative of the whole system.\n"]
["$I$18",214,"Modeling & simulations used to determine system constraints and to identify improvement opportunities. ","Purpose:  \nTo use demonstrated results from a production representative environment to verify accuracy of models/simulations and use this information to identify risk/constraints and improvement opportunities. The program needs verify the realism of the models/simulations by using actual build data from a production representative environment.  \n\n\nSources of Information:  \nThis data should come from model/simulation, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Simulation models used to determine system constraints and identify improvement opportunities\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have the modeling & simulations been used to determine system constraints?\n2. Have the modeling & simulations been used to identify improvement opportunities?\n\n\n\nAdditional Considerations:  \n•The models/simulations at this stage should be at the system level and address specific manufacturing areas to identify constraints for each prototype being built (e.g., materials, labor, equipment, tooling/Special Test Equipment (STE), setup, yield/scrap/ rework, Work In Progress (WIP), and capability/capacity).\n\n\nLessons Learned: \n•Continually update models/simulation tools as more data becomes available. \n•Accurate models and simulations will identify constraints and improvement opportunities early and provide time to work the issues that arise before transition to production.\n"]
["$J$18",215,"Modeling & simulations verified by pilot line build. Results used to improve process and demonstrate that LRIP requirements can be met. ","Purpose:  \nTo use demonstrated results from a pilot line environment to verify accuracy of models/ simulations and to use this information to identify risks/constraints and improvement opportunities. To assess whether you can meet LRIP requirements.  The program manager must verify the realism of models/simulations by using actual build data from a pilot line to ensure confidence in the ability to identify manufacturing risk/constraints and improvement opportunities.\n\n\nSources of Information:  \nThis data should come from model/simulation, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Simulation models verified by the pilot line\n•Results of the simulation models used to improve processes and determine that Low Rate Initial Production (LRIP) requirements can be met\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have the modeling & simulations been verified by the pilot line build?\n2. Have the results of pilot line modeling & simulations been used to improve process? \n3. Do the results of pilot line modeling & simulations demonstrate that Low Rate Initial Production (LRIP) requirements can be met?\n\n\n\nAdditional Considerations:  \n•Models and simulations are only useful if they reflect actual build data prior to entering LRIP.\n\n\nLessons Learned: \n•Continually update models and simulation tools as more build data becomes available, and use this information to identify risk areas as you transition into production.\n"]
["$K$18",216,"Modeling & simulations verified by LRIP build, assist in management of LRIP, and demonstrate that FRP requirements can be met.  ","Purpose:  \nTo use build data from LRIP to validate models/simulations and to use this information to identify risk/constraints and improvement opportunities. To use these tools to assess if you can meet your FRP program requirements The program must constantly verify realism in the cost and process models by using demonstrated data from a LRIP environment to ensure confidence in achieving program requirements. \n\n\nSources of Information:  \nThis data should come from modeling/simulation, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Simulation models verified by LRIP build\n•Simulation models used to determine if Full Rate Production (FRP) requirements can be met\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have modeling & simulations been verified by Low Rate Initial Production (LRIP) build.\n2. Are modeling & simulations used to assist in the management of Low Rate Initial Production (LRIP)?\n3. Do the results of Low Rate Initial Production (LRIP) modeling & simulations demonstrate that Full Rate Production (FRP) requirements can be met?  \n\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• Continually update models with actual build data and use those tools to assess future risk. "]
["$L$18",217,"Modeling & simulations verified by FRP build.  Production simulation models used as tools to assist in management of FRP.  ","Purpose:  \nTo use demonstrated results from FRP to verify you can meet FRP program requirements. To use these tools to help manage the FRP efforts. The program build data should support program requirements.  \n\n\nSources of Information:  \nThis data should come from model/simulation, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Simulation models verified by FRP build\n•Production simulation models used as a tool\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have modeling & simulations been verified by Full Rate Production (FRP) build?\n2. Are modeling & simulations used to assist in the management of Full Rate Production (FRP)?\n\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• Continually update models with actual build data and use those tools to assess future risk. "]
["$A$19",218]
["$B$19",219,"E.2 - Manufacturing Process Maturity"]
["$C$19",220,"Hypotheses developed regarding cause-effect relationships between process variables and process stability and repeatability.","Purpose:  \nTo begin identifying manufacturing requirements to build the items (e.g., material and/or process approaches), and to use this information to understand risk in pursuing this approach. To allow decision makers insight into risk and start exploring mitigation options to address those risks. The R&D community needs a basic understanding on the manufacturing requirements and the risk and implications of the approach being considered.  \n\n\nSources of Information:  \nThe S&T community, including service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  \n\n\nExamples of ‘Objective Evidence’:\n•  Hypotheses regarding cause-effect relationships between process variables and process stability and repeatability developed and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have hypotheses been developed regarding cause-effect relationships between process variables and process stability and repeatability? \n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$19",221,"Studies performed to test hypotheses regarding cause-effect relationships. Initial process approaches identified.","Purpose:  \nTo begin identifying manufacturing requirements to build the items (e.g., material and/or process approaches), and to use this information to understand risk in pursuing this approach. To allow decision makers insight into risk and start exploring mitigation options to address those risks. The R&D community needs a basic understanding on the manufacturing requirements and the risk and implications of the approach being considered.  \n\nSources of Information:  \nThe S&T community, including service laboratories and research organizations in academia and industry provide much collaborative technical information in state-of-the-art manufacturing science and technology.  Professional organizations provide the ideal forum to exchange new ideas and approaches to discovery and invention.  Gatherings of researchers from academia, industry and government, including international forums serve to further knowledge. \n\nExamples of ‘Objective Evidence’:\n•  Studies to test hypotheses regarding cause-effect relationships performed and documented\n•  Initial process approaches identified and documented\n \nQuestions (MRL Criteria in the form of a question):\n1. Have studies been performed to test hypotheses regarding cause-effect relationships? \n2. Have initial process approaches been identified?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address manufacturing risk/issues earlier in the R&D process."]
["$E$19",222,"Initial estimates of yields and rates for system concepts identified through laboratory experiments. Yield and rate gaps for system concepts identified.","Purpose:  \nTo begin identifying and understanding the capability of critical manufacturing processes of your product through experimentation, and to use this information to understand the risk in pursuing this approach. To allow decision makers insight into the manufacturing risk and start exploring mitigation options to address those risks identified.  Assessing the maturity of the manufacturing processes in developmental items mandates that the S&T and acquisition communities work together to assess manufacturing risk for every alternative being considered.\n\nSources of Information:  \nThe S&T sponsors like the Service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. At the beginning of technology development overall objectives/goals for cost, schedule and performance should be reviewed with the customers (i.e., users, laboratories, and the acquisition community) \n\nExamples of ‘Objective Evidence’:\n•  Cause-effect relationships between process control variables and process stability and repeatability through laboratory experiments have been validated and documented\n•  Critical process control variables have been identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have cause-effect relationships between process control variables and process stability and repeatability been validated through laboratory experiments? \n2. Have critical process control variables been identified?\n\nAdditional Considerations:  \n• It is especially critical that the cost goals reflect manufacturing cost considerations and capabilities.\n\nLessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address manufacturing risk/issues earlier in the R&D process."]
["$F$19",223,"Maturity of critical processes for preferred materiel solution assessed.\nProcess capability requirements and improvement plans developed and documented in the SEP. ","Purpose:  \nTo begin assessing the current demonstrated capability of critical manufacturing processes and to understand the issues/risks for supporting the AoA. To use this information to understand the manufacturing risk in selecting the alternatives to proceed to the TMRR phase. The program manager must understand the manufacturing risk of each design alternative and include that risk in program planning efforts. A critical step is to perform a capability assessment of critical processes and quantify the risk for each alternative being considered.  At the end of this phase the best available alternatives will be selected to proceed into TMRR.\n\n  \nSources of Information:  \nThis data should come from the design and manufacturing departments.\n\nExamples of ‘Objective Evidence’:\n•  Completed survey determining the current state of the maturity of critical processes for the preferred materiel solution\n•  SEP includes process capability requirements and improvement plans\n\nQuestions (MRL Criteria in the form of a question):\n1. Has the maturity of critical processes for the preferred materiel solution been assessed?\n2. Have process capability requirements and improvement plans been developed and documented in the SEP?\n\nAdditional Considerations:  \n•What are the critical processes that need attention at this stage of the product life-cycle?\n\nLessons Learned: \n•We must assess current critical process capability versus that required and use the data in AoAs and in the ASRs.\n"]
["$G$19",224,"Process maturity assessed on similar processes in production.  Process capability requirements identified for pilot line, LRIP and FRP.","Purpose:  \nTo obtain more fidelity assessing process capability by using data from building prototypes and similar processes to assess whether target costs are achievable. Data generated from this assessment will be used to make design choices, make/buy decisions, and capacity, process capability, sources, quality, key characteristics, yield/rate, and variability assessments.  The program manager must continue to assess current process capability versus what is needed to meet program requirements.  Using actual build data can ensure adequate assessment of manufacturing risk transitioning to production.  At the end of this phase you will select the best alternatives to proceed into EMD.\n\n\nSources of Information:  \nThis data should come from design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Process maturity assessed on similar processes in production documented\n•Process capability requirements identified and documented for pilot line, Low Rate Initial Production (LRIP) and Full Rate Production (FRP)\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has process maturity been assessed on similar processes currently in production?\n2. Have process capability requirements been identified for pilot line, Low Rate Initial Production (LRIP) and Full Rate Production (FRP)?\n\n\nAdditional Considerations:  \n•You should understand risk that each selected alternative presents--especially for critical processes.\n\n\nLessons Learned: \n•The need for realism in the process capability assessment requires using actual hardware build data as soon as possible.\n"]
["$H$19",225,"Manufacturing processes demonstrated in production relevant environment.  Collection or estimation of process capability data from prototype build and refinement of process capability requirements initiated.","Purpose:  \nTo demonstrate process capability in a production relevant environment. To obtain more fidelity in assessing process capability by using data from building prototypes of from similar production processes to determine whether program objectives are achievable. Data generated from this assessment will be used to make design choices, make/buy decisions, and capacity, process capability, sources, quality, key characteristics, yield/rate, and variability assessments.  The program is required to produce in a production relevant environment and the data will compare existing process capability versus program requirements.  Using actual prototype build data in performing these assessments will ensure adequate assessment of risk transitioning to production.  At the end of this phase you will select the best alternatives available to proceed into EMD.  \n\n\nSources of Information:  \nThis data should come from cost design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Process demonstration documentation\n•  Process capability data from prototype build documented\n•  Process capability requirements documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have the manufacturing processes been demonstrated in a production relevant environment?\n2.  Has collection or estimation of process capability data from prototype builds been initiated?\n3.  Has refinement of process capability requirements from prototype builds been initiated?\n\n \nAdditional Considerations:  \n•  You should understand risk that each selected alternative presents--especially for critical processes.\n\n\nLessons Learned: \n•  The need for realism in the process capability assessment requires using actual hardware build data as soon as possible.\n"]
["$I$19",226,"Manufacturing processes demonstrated in a production representative environment. Collection and/or estimation of process capability data and refinement of process capability requirements ongoing.","Purpose:  \nTo demonstrate process capability in a production representative environment. To obtain more fidelity in assessing process capability by using data from building EMD units and similar processes in production to determine whether program objectives are achievable. Data generated from this assessment will be used to make design choices, make/buy decisions, and capacity, process capability, sources, quality, key characteristics, yield/rate, and variability assessments. At this phase the data must ensure process capability to meet program requirements. It is essential to use real data performing these assessments to ensure adequate risk assessment of transitioning to production.  At the end of this phase you will transition into LRIP.  \n\n\nSources of Information:  \nThis data should come from cost design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Manufacturing processes successfully demonstrated in a production representative environment \n•Collecting or estimating process capability data is continuing\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have the manufacturing processes been demonstrated in a production representative environment?\n2. Is collection and/or estimation of process capability data (i.e., from lower level sub-systems, items, and components) ongoing?\n3. Is refinement of requirements at all levels for process capabilities ongoing?\n\n\n\nAdditional Considerations:  \n•What impact will design changes have on process capability if different or changed processes are required?\n•Taguchi Design of Experiments (DOE) is an effective approach to control process variability.\n\n\nLessons Learned: \n•Continually assess demonstrated process capability as more data becomes available, and compare it to production requirements.\n"]
["$J$19",227,"Manufacturing processes for LRIP verified on a pilot line.  Process Capability data from pilot line meets target. Process capability requirements for LRIP and FRP refined based upon pilot line data.","Purpose:  \nTo demonstrate that manufacturing process capabilities are proven to be adequate in a pilot line production environment, to ensure ability to achieve Low Rate Initial Production objectives. A Pilot line demonstration provides verifiable process capability/ performance results to compare to LRIP requirements. Some of the data includes information to determine whether tolerances can be achieved for all Key Characteristics.  It is essential to use actual pilot line build data to ensure manufacturing risk of transitioning into production has been adequately assessed.    \n\n\nSources of Information:  \nThe majority of the results from your Pilot Line will come from the manufacturing/industrial engineering and quality departments. Some of key measures will be process capability charts, cycle time, total manufacturing hours, scrape/rework percentages, first-pass-yield results, realization factors from actual versus standards, and change rates (Design and Processes).\n\n\nExamples of ‘Objective Evidence’:\n•Manufacturing processes for Low Rate Initial Production (LRIP) verified on a pilot line\n•Capability data from the pilot line meets the target \n•Process capability requirements for LRIP refined based on Pilot Line data\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have the manufacturing processes to be used for Low Rate Initial Production (LRIP) been verified on a pilot line?\n2. Does the data collected from pilot line builds meet the process capability (i.e., Cpk) targets? \n3. Based on the data collected from pilot line builds, have the requirements for process capabilities (i.e., Cpk) been refined for LRIP and FRP? \n\n\n\nAdditional Considerations:  \n•Taguchi Design of Experiments (DOE) is an effective approach to control process variability.\n\n\nLessons Learned: \n•One of the important steps for achieving this requirement is to work with the contractor/ supplier on what represents a pilot production line for your programs. This effort should be done as soon as practical and if at all possible, before the award of the contract for EMD. \n•This difficult task will require SMEs to help define the requirements that best support your unique program requirements. \n•What future rate and performance requirements and what critical processes need to be monitored on the Pilot line to ensure LRIP cost, schedule, and performance objectives will be met. \n•Focusing in on process capability measures via Statistical Process Control (SPC) may not be possible due to small lot sizes; therefore, look at other key performance output measures to determine the process capability (e.g., first pass yield, scrap/rework, cycle time, etc.).\n"]
["$K$19",228,"Manufacturing processes are stable, adequately controlled, capable, and have achieved program LRIP objectives.  Variability experiments conducted to show FRP impact and potential for continuous improvement. ","Purpose:  \nTo use demonstrated results from a LRIP line to verify processes are stable, adequately controlled, and capable. To use build data to analyze the process capability to identify areas for improvement. The program needs to constantly assess the process capability by using actual build data from a LRIP environment to ensure confidence in meeting program objectives. This early production data will set the foundation for future production cost and performance.  \n\n\nSources of Information:  \nThis data should come from design and manufacturing departments.\n \n\nExamples of ‘Objective Evidence’:\n•Manufacturing processes demonstration documented\n•Process capability documentation\n•variability experiments documentation\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are manufacturing processes stable, capable, and adequately controlled (i.e., meeting Cpks) for Low Rate Initial Production (LRIP)?\n2. Have manufacturing processes achieved program Low Rate Initial Production (LRIP) objectives? \n3. Have variability experiments been conducted to show impacts from Full Rate Production (FRP)?\n4. Have variability experiments been conducted to determine the potential for continuous improvement?  \n\n\nAdditional Considerations:  \n• Taguchi Design of Experiments (DOE) is an effective approach to control process variability.\n\n\nLessons Learned: \n• Continually update process capability assessments as more data becomes available and use this data to determine the likelihood of achieving program objectives."]
["$L$19",229,"Manufacturing processes are stable, adequately controlled, capable, and have achieved program FRP objectives.  ","Purpose:  \nTo demonstrate process capability to meet FRP objectives. The program actual build data must support program objectives.\n\n\nSources of Information:  \nThis data should come from cost design and manufacturing departments.\n \n\nExamples of ‘Objective Evidence’:\n•Process demonstration documented\n•Process capability requirements documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are manufacturing processes stable, capable, and adequately controlled (i.e., meeting Cpks)?\n2. Have manufacturing processes achieved program rate production (FRP) objectives?\n\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• Continually update process capability assessments as more data becomes available and use this data to determine the likelihood of continually meeting program objectives. "]
["$A$20",230]
["$B$20",231,"E.3 - Process Yields and Rates"]
["$C$20",232,"Hypotheses developed regarding future state manufacturing yields and rates.","Purpose:  \nTo develop hypotheses on the yields and rates needed for concepts. To effectively transition R&D projects one should address the manufacturing maturity of the concepts under development. The actual yields achieved on these projects will provide both the S&T and acquisition community a good indicator of the process maturity and the risk of proceeding. \n\n\nSources of Information:  \nThe S&T sponsors like the Service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. At the beginning of development or when it is known that a project will transition to acquisition, both S&T and acquisition communities should have an overall objectives/goals for cost, schedule and performance that will be demonstrated. These objectives should be reviewed with the customers (i.e., users, S&T, and Acquisition communities) on some regular schedule.  It is especially critical that cost goals reflect manufacturing considerations and capabilities.\n\n\nExamples of ‘Objective Evidence’:\n•  Hypotheses regarding future state manufacturing yields and rates developed and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have hypotheses been developed regarding future state manufacturing yields and rates?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$20",233,"Studies performed to test hypotheses regarding yields and rates.","Purpose:  \nTo develop hypotheses on the yields and rates needed in R&D. To effectively transition R&D projects one should address the manufacturing maturity of the concepts under development. The actual yields achieved on these projects will provide both the S&T and acquisition community a good indicator of the process maturity and the risk of proceeding. \n\nSources of Information:  \nThe S&T sponsors like the Service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. At the beginning of development or when it is known that a project will transition to acquisition, both S&T and acquisition communities should have an overall objectives/goals for cost, schedule and performance that will be demonstrated. These objectives should be reviewed with the customers (i.e., users, S&T, and Acquisition communities) on some regular schedule.  It is especially critical that cost goals reflect manufacturing considerations and capabilities.\n\nExamples of ‘Objective Evidence’:\n•  Studies to test hypotheses regarding yields and rates performed and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have studies been performed to test hypotheses regarding yields and rates? \n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$E$20",234,"Initial estimates of yields and rates based on experiments or state of the art.","Purpose:  \nTo a get a baseline on the yields and rates demonstrated in R&D. To effectively transition R&D projects one should address the manufacturing maturity of the prototypes being developed. The actual yields achieved on these projects will provide both the S&T and acquisition community a good indicator of the process maturity and the risk of proceeding.\n\nSources of Information:  \nThe S&T sponsors like the Service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. At the beginning of development or when it is known that a project will transition to acquisition, both S&T and acquisition communities should have an overall objectives/goals for cost, schedule and performance that will be demonstrated. These objectives should be reviewed with the customers (i.e., users, S&T, and Acquisition communities) on some regular schedule.  It is especially critical that cost goals reflect manufacturing considerations and capabilities.\n\nExamples of ‘Objective Evidence’:\n•  Initial estimates of yields and rates for system concepts through laboratory experiments have been identified and documented\n•  Yield and rate gaps for system concepts have been identified and documented \n\nQuestions (MRL Criteria in the form of a question):\n1. Have initial estimates of yields and rates for system concepts been identified through laboratory experiments? \n2. Have yield and rate gaps for system concepts been identified? \n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address manufacturing maturity and yield issues earlier in the R&D process."]
["$F$20",235,"Yield and rate assessments on preferred materiel solution completed and considered in the AoA. Yield and rate gap closure strategies identified for the preferred materiel solution and documented in the SEP.","Purpose:  \nTo understand the yields and rates achieved on the R&D project or on similar process used by this contractor to assess the manufacturing risk of proceeding to Milestone A and to reflect that risk in the AoA. The yields and rates that have been demonstrated to date, either through R&D or a similar process, need to be compared with program requirements. This takes some judgment and should be done carefully; however, it is an excellent indicator of program risk.\n\nSources of Information:  \nThis data should come from quality, design, and manufacturing departments.\n\nExamples of ‘Objective Evidence’:\n•  Completed yield and rates assessment on proposed/similar processes\n•  Yield and rates assessment on proposed/similar processes applied within the Analysis of Alternatives (AoA) \n•  SEP includes yield and rate gap closure strategies for the preferred materiel solution\n\nQuestions (MRL Criteria in the form of a question):\n1. Have yield and rate assessments on the preferred materiel solution been completed and considered in the AoA?\n2. Have yield and rate gap closure strategies been identified for the preferred materiel solution and documented in the SEP?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address manufacturing risk/issues earlier in the R&D process.\n• The yields being demonstrated versus what are required is a best indicator to provide the current maturity and risk of proceeding forward.\n• If yields are below 60% on critical processes at this stage, you need to monitor this process very closely.\n"]
["$G$20",236,"Target yields and rates established for pilot line, LRIP, and FRP. Yield and rate issues identified. Improvement plans developed/initiated. ","Purpose:  \nTo understand the current yields and rates and develop targets for growth of these factors at key milestones in the development cycle (e.g., pilot line, LRP, and FRP). To ensure improvements plans have been developed and are being implemented to achieve these growth targets. The program manager must assess the current yields and rates against program requirements and implement plans to achieve required targets.  Further, if there is no other option, the progress in improving your yields and rates may be the key factor in determining program success.\n\n\nSources of Information:  \nThis data should come from cost quality, design, and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Target yield and rates established and documented for the pilot line, Low Rate Initial Production (LRIP), and Full Rate Production (FRP)\n•  Detailed production schedule supports the required production rate \n•  Plans have been developed/initiated for yield and rate improvement\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have target yield and rates been established for the pilot line, Low Rate Initial Production (LRIP), and Full Rate Production (FRP)?\n2.  Have any risks and issues arising from yields and rates been identified?\n3.  Have improvement plans for yields and rates been developed and initiated? \n\n\nAdditional Considerations:  \n•  It is important at this phase to develop process improvement plans where design trade-offs and process selection can be made easier to achieve program requirements.\n\n\nLessons Learned: \n•  The yields being demonstrated versus what are required is a best indicator to provide the current maturity and risk of proceeding forward.\n•  If yields are below 60% on critical processes at this stage, you need to monitor this process very closely.\n"]
["$H$20",237,"Yields and rates from production relevant environment evaluated against targets and the results feed improvement plan.","Purpose:  \nTo assess the yields and rates achieved building prototypes in a production relevant environment, compare them to targets, and determine the manufacturing risk of meeting future requirements. To use this data to make the appropriate mitigation/improvement plans to achieve future requirements.  The program manager must understand current yields and rates and compare those to requirements. If this is not addressed before EMD it will become a major problem. \n\n\nSources of Information:  \nThis data should come from quality, design, and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Yields and rates and targets documented and evaluated against targets\n•  Improvement plans reflect results from the yields and rates from the production relevant environment\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Are yields and rates from the production relevant environment being evaluated against production representative environment targets?\n2.  Are results of the evaluation of production relevant yields and rates included in the ongoing improvement plans?\n\n\nAdditional Considerations:  \n•  Exercise caution, because even if you meet yield requirements in pre-EMD, the risk could still be high for production quantities.\n\n\nLessons Learned: \n•  The yields being demonstrated versus what are required is a best indicator to provide the current maturity and risk of proceeding forward.\n•  If yields are below 60% on critical processes at this stage, you need to monitor this process very closely.\n•  This is a good to time to assess design requirements versus process capability to see if opportunities exist to improve yields and rates without compromising performance requirements.\n"]
["$I$20",238,"Yields and rates from production representative environment evaluated against pilot line targets and the results feed improvement plans.","Purpose:  \nTo use results from a production representative environment to demonstrated that you can achieve pilot line requirements for yields and rates. To use your data for updating your mitigation/improvement plans required to achieve targeted yields and rate requirements. The program needs to verify the likelihood of achieving production requirements from demonstrated data built in a production representative environment. The program needs data to make updates to improvement plans. \n\n\nSources of Information:  \nThis data should come from quality, design, and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Yields and rates from the production representative environment evaluated against pilot line targets\n•Improvement plans reflect results from the yields and rates from the production representative environment\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are yields and rates from the production representative environment being evaluated against pilot line targets? \n2. Are results of an evaluation of the production representative yields and rates included in the ongoing improvement plans? \n\n\n\nAdditional Considerations:  \n•Exercise caution, because even if you meet yield requirements in EMD, the risk could still be high for full-rate production quantities.\n\n\nLessons Learned: \n•Continually update yield and rate targets as more data becomes available and use the data to make informed program decisions and process improvements. \n•Not achieving targets by this time will demonstrate the need to closely monitor this effort and identify mitigations to minimize the impact of not achieving your targets.\n"]
["$J$20",239,"Pilot line targets achieved. Yields and rates required to begin LRIP refined using pilot line results. Improvement plans ongoing and updated.","Purpose:  \nTo use data from the pilot line environment to demonstrate that yields and rates will support production requirements. To implement a continuous improvement program using yield and rate data. The program needs verify the realism of achieving production requirements by using build data from a pilot line environment. To ensure we have confidence in our ability to achieve program requirements we should have achieved our production yield and rate targets at this time.\n\n\nSources of Information:  \nThis data should come from quality, design, and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Pilot line yield and rate targets achieved\n•Yields and rates required to begin Low Rate Initial Production (LRIP) are refined using pilot line articles actual data\n•Yield and rate improvement plans are being updated based on actual data\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have pilot line yields and rates been achieved?\n2. Are the results of an evaluation of yields and rates from the pilot line being used to refine Low Rate Initial Production (LRIP) requirements?\n3. Are yield and rate improvement plans ongoing and up-to-date?\n\n\n\nAdditional Considerations:  \n•Exercise caution, because even if you meet yield requirements in EMD, the risk could still be high for full-rate production quantities.\n•The contractor should implement a continuous improvement program using data generated from the pilot line.\n\n\nLessons Learned: \n•Continually update yield and rate targets as more data becomes available and use the data to make informed program decisions and process improvements.\n"]
["$K$20",240,"LRIP yield and rate targets achieved.  Yields and rates required to begin FRP refined using LRIP results. Yield improvements on-going. ","Purpose:  \nTo use demonstrated results from a LRIP line to verify you have achieved the target yield and rates to meet production rate requirements.  To implement a continuous improvement program using yield and rate data. The program manager must verify the likelihood of achieving production requirements by using actual build data from LRIP. To ensure you have confidence in your ability to achieve program requirements you should have achieved either FRP production yield and rate targets or have a low risk plan in place to achieve them by this time.  \n\n\nSources of Information:  \nThis data should come from quality, design, and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Yields and rates and targets documented and evaluated against targets\n•Improvement plans reflect results from the yields and rates from LRIP\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have the Low Rate Initial Production (LRIP) yield and rate targets been achieved? \n2. Are the results of an evaluation of yields and rates from Low Rate Initial Production (LRIP) being used to refine rate production (FRP) requirements?\n3. Are yield and rate improvements on-going?\n\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• Continually update yield and rate targets as more data becomes available and use the data to make informed program decisions and process improvements."]
["$L$20",241,"FRP yield and rate targets achieved.  Yield improvements on-going. ","Purpose:  \nTo verify FRP yield and rate targets have been achieved. The program manager must verify achieving production requirements by using the actual build data. To ensure confidence in our ability to achieve program requirements we should have achieved FRP production yield and rate targets or have a low risk plan in place to achieve them by this time. \n\n\nSources of Information:  \nThis data should come from quality, design, and manufacturing departments.\n \n\nExamples of ‘Objective Evidence’:\n•Yields and rates and targets achieved in FRP\n•Yield improvement plans documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have the requirements for Full Rate Production (FRP) yields and rates been met?\n2. Are yield improvements on-going?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• Continually update yield and rate targets as more data becomes available and use the data to make continuous process improvements."]
["$A$21",242,"F - Quality Management","Quality Management: Requires an analysis of the risks and management efforts to control quality, and foster continuous improvement at prime and suppliers.\n\nQuality Management includes supplier quality."]
["$B$21",243,"F.1 - Quality Management."]
["$C$21",244,"Quality management considerations surveyed and included in early planning activities.","Purpose: \nTo lay the groundwork for ongoing Quality Management System development concurrent with concept exploration.  To consider potential effects of quality on new products or manufacturing technologies and any technology development related to quality processes in the concept phase. Quality management consideration for concepts are surveyed.\n\n\nSources of Information:\nAt this stage in development most useful information will come from historical data on other similar programs.  Quality successes and issues dealt with in the past on programs that have common or similar manufacturing processes will provide valuable lessons to apply in planning a Quality Strategy.  \n\n\nExamples of ‘Objective Evidence’:\n•  Quality management considerations surveyed and included in early planning activities \n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have quality management considerations been surveyed and included in early planning activities? \n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$21",245,"Quality management needs assessed, analyzed, and validated.","Purpose: \nTo lay the groundwork for ongoing Quality Management System development concurrent with system design, in parallel with the development of a manufacturing capability for system concepts.  To document potential effects of quality on new products or manufacturing technologies and any technology development related to quality processes in the concept phase. Quality management needs for system concepts are documented.\n\nSources of Information:\nAt this stage in development most useful information will come from historical data on other similar programs.  Quality successes and issues dealt with in the past on programs that have common or similar manufacturing processes will provide valuable lessons to apply in planning a Quality Strategy.  Design and technology reviews are primary source of program specific information at this phase and, as the design further develops, technologies that affect manufacturing and quality will become apparent.\n\nExamples of ‘Objective Evidence’:\n•  Quality management needs for system concepts assessed, analyzed, and validated and documented\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have quality management needs been assessed, analyzed, and validated?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$E$21",246,"Quality management requirements for system concepts identified.","Purpose: \nTo lay the groundwork for ongoing Quality Management System development concurrent with system design, in parallel with the development of a manufacturing capability for the system.  To document potential effects of quality on new products or manufacturing technologies and any technology development related to quality processes in the Technology Development Strategy. Quality management requirements for system concepts are documented.\n\nSources of Information:\nAt this stage in development most useful information will come from historical data on other similar programs.  Quality successes and issues dealt with in the past on programs that have common or similar manufacturing processes will provide valuable lessons to apply in planning a Quality Strategy.  Design and technology reviews are primary source of program specific information at this phase and, as the design further develops, technologies that affect manufacturing and quality will become apparent.\n\nExamples of ‘Objective Evidence’:\n•  Quality management requirements for system concepts have been identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have quality management requirements for system concepts been identified?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$F$21",247,"Quality strategy for the preferred materiel solution developed, considered in the AoA, and documented in the SEP and the AS. ","Purpose: \nTo lay the groundwork for ongoing Quality Management System development concurrent with system design, in parallel with the development of a manufacturing capability for the system.  To document potential effects of quality on new products or manufacturing technologies and any technology development related to quality processes in the Technology Development Strategy.  Assessment of manufacturing feasibility in the AoA is a requirement in the MSA phase for each of the Critical Technologies (CTs) (a.k.a, CTEs), from the TRA, in all possible material solutions.  Manufacturing process yields and inspectability may be relevant considerations for some or all material solutions.  \n\n\nSources of Information:\nAt this stage in development most useful information will come from historical data on other similar programs.  Quality successes and issues dealt with in the past on programs that have common or similar manufacturing processes will provide valuable lessons to apply in planning a Quality Strategy.  Design and technology reviews are primary source of program specific information at this phase and, as the design further develops, technologies that affect manufacturing and quality will become apparent.\n\nExamples of ‘Objective Evidence’:\n•  Quality strategy for the preferred materiel solution documented\n•  Quality strategy identified and documented for the preferred materiel solution in the AoA\n•  Quality strategy documented in the System Engineering Plan (SEP) and contractor System Engineering Management Plan (SEMP)\n•  Quality strategy documented in the AS\n\nQuestions (MRL Criteria in the form of a question):\n1. Has the quality strategy for the preferred materiel solution been developed?\n2. Has the quality strategy for the preferred materiel solution been considered in the AoA?\n3. Has the quality strategy for the preferred materiel solution been documented in the SEP and the AS?\n\nAdditional Considerations:\n• Is there a plan to evaluate the quality strategy as part of Analysis of Alternatives (AoA)? \n• What data is available from similar, more mature, systems to assist in quality planning?\n• Are there any lessons learned on quality strategies from other similar system?\n• To properly assess risks and compare alternatives the effect of quality on cost, schedule and performance should not be ignored. \n\nLessons Learned: \n• Developing a Quality Strategy as early as possible in the lifecycle prevents potential misunderstandings about quality requirements.   Identifying a need for a strategy addressing quality in specific phases, manufacturing or assembly conditions, or system components within an outlined quality strategy pre Milestone A will ensure the details are well considered and updated later in Engineering and Manufacturing Development (EMD).\n• Many program managers and chief engineers will have difficulty recognizing the need to address quality strategy this early in the program.  Make sure they are brought up to speed on the logic for this step in the process, and keep them informed as the strategy is developed so they see the value added.\n"]
["$G$21",248,"Quality strategy updated to reflect KC identification activities.  ","Purpose:  \nTo build on the groundwork already established for an ongoing Quality Management System development concurrent with the system design in parallel with the development of a manufacturing capability for the system.  Consideration should be given to the compatibility of the existing Quality Management System with the quality requirements likely to apply to the product in question (e.g., ISO9001 or AS9100).  The potential effect of quality on new product or manufacturing technology and any technology development related to quality processes are documented in the Technology Development Strategy.  The basic outline for the quality system is documented in the Systems Engineering Plan. Assessment of manufacturing feasibility in the AoA is a requirement for each of the Critical Technologies (CTs), from the TRA, in all possible material solutions.   Further evaluation of technology demonstrations in the Technology Development phase provides greater opportunity to identify quality risks and opportunities to improve future process yields.  Manufacturing process yields and inspectability may be relevant considerations for some or all material solutions.  \n\n\nSources of Information: \nAt this stage in development most useful information will come from historical data on other similar programs.  Quality successes and issues dealt with in the past on programs that have common or similar manufacturing processes will provide valuable lessons to apply in planning a Quality Strategy.  Design and technology reviews are primary source of program specific information at this phase and, as the design further develops, technologies that affect manufacturing and quality will become apparent.\n\n\nExamples of ‘Objective Evidence’:\n•  Quality strategy documented and updated to reflect Key Characteristic identification activities \n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Does the quality strategy include activities for Key Characteristic identification?\n\n\nAdditional Considerations:\n•  Is there a plan to evaluate the quality strategy as part of Analysis of Alternatives (AoA)? \n•  What data is available from similar, more mature, systems to assist in quality planning?\n•  Are there any lessons learned on quality strategies from other similar system?\n•  To properly assess risks and compare alternatives, the effect of quality on cost, schedule and performance should not be ignored.\n\n\nLessons Learned: \n•  Developing a Quality Strategy as early as possible in the lifecycle prevents potential misunderstandings about quality requirements.   Identifying a need for a strategy addressing quality in specific phases, manufacturing or assembly conditions, or system components within an outlined quality strategy pre-Milestone A will ensure the details are well considered and updated later in Engineering and Manufacturing Development (EMD).\n•  Many program managers and chief engineers will have difficulty recognizing the need to address quality strategy this early in the program.  Make sure they are brought up to speed on the logic for this step in the process, and keep them informed as the strategy is developed so they see the value added.\n•  Per Quality Management Systems requirements (e.g., AS9100), the Quality Manager is to report to the ‘Executive Level’ within companies.\n"]
["$H$21",249,"Initial Quality Plan and QMS are in place.  Quality risks, issues, and metrics have been identified and improvement plans initiated. ","Purpose: \nTo establish criteria (thresholds) for evaluating Quality Management System(s) going into design and development reviews during EMD.  To continue to develop an ongoing Quality System concurrent with the system design, and in parallel with the development of a manufacturing capability for the system.  Since the Quality Management System utilized by the contractor is often based on AS 9100 or ISO9001, consideration should be given to compatibility of the existing Quality System with the quality requirements coming out of design development.  Program Quality Plans are also created to document the integration of the program office’s quality processes with the contractor’s, depot’s, subcontractor’s and DCMA’s Quality systems. The identification of manufacturing risks and demonstration of processes in a production relevant environment to assure an affordable program is an exit requirement for the Technology Development phase.  The Quality Management System is critical to evaluation of manufacturing performance, and costs associated with poor quality management can make a program unaffordable; therefore, preparing for the development of an effective Quality Management System is a critical element leading into EMD and milestone B.  Inspectability may also be a consideration since failure to identify defects increases cost throughout the lifecycle of the system as are support costs driven by poor reliability caused by latent defects and uncontrolled manufacturing process variability.  \n\n\nSources of Information: \nAt this stage in development most useful information will come from historical data on other similar programs.  Quality successes and issues dealt with in the past on programs that have common or similar manufacturing processes will provide valuable lessons to apply in planning a Quality Strategy.  Design and technology reviews are primary source of program specific information at this phase and, as the design further develops, technologies that affect manufacturing and quality will become apparent.\n\n\nExamples of ‘Objective Evidence’:\n•  Quality Plan and a Quality Management System documented\n•  Quality risks and metrics identified and improvement plans initiated\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Is an initial Quality Plan (QP) in place?\n2.  Is a Quality Management System (QMS) in place?\n3.  Have risks and issues impacting quality been identified?\n4.  Have quality metrics been identified?\n5.  Have improvement plans to mitigate quality risks and issues been initiated?\n\n\nAdditional Considerations:\n•  What data is available from similar, more mature, systems to assist in quality planning?\n•  Does data from similar systems provide confidence in the QMS to be used on this program?\n•  Are there any lessons learned on quality strategies from other similar system?\n•  To properly assess risks in EMD, the effect of quality on cost, schedule and performance should not be ignored.\n\n\nLessons Learned: \n•  Leading into milestone B, there will be greater attention to quality system development than during previous MRL Assessments.  Make optimum use of plans and information gleaned during earlier reviews, and make sure the plans for quality system improvement address all shortfalls and will result in a capable quality system by the end of EMD.\n•  The ability to identify quality system shortfalls will depend on how much insight management gets from the review of quality metrics.  Make sure the quality metrics package is complete, accurate, and timely to the needs of management.  Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions.\n•  Per Quality Management Systems requirements (e.g., AS9100), the Quality Manager is to report to the ‘Executive Level’ within companies.\n"]
["$I$21",250,"Quality targets established. QMS elements (i.e., control of nonconforming material, corrective action, etc.) meet requirements of appropriate industry standards. Program-specific Quality Program Plan developed.   ","Purpose: \nTo apply the quality targets/thresholds to pre-production data to evaluate Quality Management System(s) coming out of milestone B and going into design and development reviews during EMD.  To continue to develop an ongoing Quality Management System concurrent with the system design, and in parallel with the development of a manufacturing capability for the system.  Utilizing AS 9100 or ISO9001, to ensure the QMS meets industry standards applicable to the aerospace and defense system is being developed.  The basic outline for the quality system is documented in the Systems Engineering Plan, Program Quality Plans, Contractor(s) Quality Plans, and Memorandums of Understanding (MOUs) with DCMA and maintenance depots.  Demonstration of a capable manufacturing system is required during EMD, and the QMS is an important element of the manufacturing system.   The QMS is critical to evaluation of manufacturing performance, and the costs associated with poor quality management can make a program unaffordable.  \n\n\nSources of Information: \nAt this stage in the acquisition process some information from pre-production and low rate production units may be available.  Other useful information will come from historical data on other similar programs.  Preliminary Design Review assessment of feasibility is a primary source of program specific information at this phase and, as the design further develops, technologies that affect manufacturing and quality will become apparent.\n\n\nExamples of ‘Objective Evidence’:\n•Quality Management System elements (e.g., control of nonconforming material, corrective action, etc.) meet requirements of appropriate industry standards\n•Program-specific Quality Plan being developed\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are yields and rates from the production representative environment being evaluated against pilot line targets? \n2. Are results of an evaluation of the production representative yields and rates included in the ongoing improvement plans? \n\n\n\nAdditional Considerations:\n•What quality targets are in place, and what is your status in achieving these targets?\n•Does data from similar systems provide confidence in the QMS which will be used on this program?\n•What progress has been made with regard to quality improvement initiatives?\n\n\nLessons Learned: \n•Coming out of TMRR Phase, there should be some momentum regarding quality system development highlighted during the previous MRL Assessments.  Schedule regular reviews of progress to maturation plans and plans for quality system improvement to ensure the momentum isn’t lost after the milestone review.  These quality improvements will help ensure a capable quality system by the end of EMD.\n•The ability to manage program quality will depend on how much insight management gets from the review of quality metrics.  Make sure the quality metrics package is complete, accurate, and timely to the needs of management.  Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions.\n•Per Quality Management Systems requirements (e.g., AS9100), the Quality Manager is to report to the ‘Executive Level’ within companies.\n"]
["$J$21",251,"Program-specific Quality Program Plan established. Program Quality Manager assigned. Quality targets assessed against pilot line, results feed continuous quality improvements. ","Purpose: \nTo apply the quality targets/thresholds to pilot line data to evaluate the Quality Management System(s) coming out of EMD.  Continue to develop an ongoing Quality System concurrent with the system design, and in parallel with the development of a manufacturing capability for the system.  Utilizing AS 9100 or ISO9001, to ensure the QMS meets industry standards applicable to the aerospace and defense system is being developed.  The basic outline for the quality system is documented in the Systems Engineering Plan, Program Quality Plans, Contractor(s) Quality Plans, and Memorandums of Understanding (MOUs) with DCMA and maintenance depots.  Quality data should begin to make the case that maturity of the production system will soon justify entrance into LRIP, and then Full Rate Production.  Demonstration of a capable manufacturing system is required during EMD, and the QMS is an important element of the manufacturing system.  The QMS is critical to evaluation of manufacturing performance, and the costs associated with poor quality management can make a program unaffordable; therefore, the development of an effective Quality Management System is a critical element in EMD.  \n\n\nSources of Information: \nAt this stage in the acquisition process some information from production representative lines and pilot lines should be available.  Critical Design Review assessment of system development is a primary source of program specific information at this phase and, as the design further develops, manufacturing and quality systems will become well defined and proven, to allow entrance into LRIP and Full Rate Production.\n\n\nExamples of ‘Objective Evidence’:\n•Program-specific quality plan and Quality Manager established and documented\n•Quality targets have been assessed against the pilot line, and quality target assessments drive continuous quality improvement\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has the program-specific Quality Program Plan been established?\n2. Has a program-specific Quality Manager been assigned?\n3. Have the previously established quality targets for pilot line been met? \n4. Have results of an evaluation of pilot line quality data actuals against quality targets been used for continuous quality improvement? \n\n\n\nAdditional Considerations:\n•What quality targets are in place, and what is your status in achieving these targets?\n•Does quality data from production representative and pilot line units provide confidence in the QMS which will be used on this program?\n•Is the QMS compliant with AS 9100 or ISO9001?\n\n\nLessons Learned: \n•Preparing for milestone C, there should be a great deal of program management attention to quality system development and resolution of issues identified during the CDR and previous MRL Assessments.  Continue to schedule regular reviews of progress to maturation plans and plans for quality system improvement to ensure the program won’t experience any surprises going into milestone C.  These quality improvements will help ensure a capable quality system for LRIP and Full Rate Production.\n•The ability to manage program quality will depend on how much insight management gets from the review of quality metrics.  Make sure the quality metrics package is complete, accurate, and timely to the needs of management.  Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions.\n•Per Quality Management Systems requirements (e.g., AS9100), the Quality Manager is to report to the ‘Executive Level’ within companies.\n"]
["$K$21",252,"Quality targets verified on LRIP line. Continuous quality improvement on-going.  Management review of Quality measures conducted on regular basis and appropriate actions taken.","Purpose: \nTo apply quality targets/thresholds to LRIP data to evaluate the Quality Management System(s) coming out of Milestone C and to demonstrate the capability of the production and quality systems to handle full rate production.  AS9100 is an industry standard (along with other industry standards) that are applicable to our QMS.  The basic outline for the quality system is documented in the Systems Engineering Plan, Program Quality Plans and Contractor(s) Quality Plans. Quality data should indicate the level of maturity of the production system that will soon justify entrance into Full Rate Production.  Manufacturing processes must be under control at milestone C.  During LRIP final manufacturing development will be completed, and the QMS is an important element of the manufacturing system.  The QMS is also critical to evaluation of manufacturing performance, and the costs associated with a poor quality management system can make a program unaffordable.  Therefore, the development of an effective Quality Management System is a critical element in Production.    \n\n\nSources of Information: \nAt this stage in the acquisition process data from production representative lines and pilot lines should be available and some data from LRIP units will begin to come in.  Milestone C assessment of system development, including the MRL Assessment leading into the milestone review, is a primary source of program specific information at this phase.  The manufacturing and quality systems should be defined and proven in order to allow entrance into Full Rate Production.\n\n\nExamples of ‘Objective Evidence’:\n•Quality Plan and a Quality Management System documented\n•Quality targets verified during LRIP\n•Continuous quality improvement activity evident\n•Management review of quality targets and appropriate actions evident\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have quality targets been verified on the Low Rate Initial Production (LRIP) line?\n2. Is continuous quality improvement on-going? \n3. Is management review of quality measures conducted on a regular basis with appropriate actions being taken?\n\n\n\nAdditional Considerations:\n•What quality targets are in place, and what is your status in achieving these targets?\n•Does quality data from LRIP units provide confidence in the QMS which will be used on this program?\n•Is the QMS compliant with AS9100 or another Industry Standard?\n\n\nLessons Learned: \n• Preparing for Full Rate Production, there should be a great deal of program management attention to quality system development and resolution of issues identified during the previous MRA, CDR, and milestone reviews.  Continue to schedule regular reviews of progress to maturation plans and plans for quality system improvement to ensure the program won’t experience any surprises going into Full Rate Production.  These quality improvements will help ensure a capable quality system for LRIP and Full Rate Production.\n• The ability to manage program quality will depend on how much insight management gets from the review of quality metrics.  Make sure the quality metrics package is complete, accurate, and timely to the needs of management.  Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions.\n•Per Quality Management Systems requirements (e.g., AS9100), the Quality Manager is to report to the ‘Executive Level’ within companies."]
["$L$21",253,"Quality targets verified on FRP line. Continuous quality improvement on-going.  Statistical controls applied where appropriate.  ","Purpose: \nTo continuously improve quality in support of, and throughout full rate production.  LRIP data out of the QMS helps identify opportunities for continuous improvement.  Quality data should indicate the level of maturity of the production system is commensurate with a program in Full Rate Production.   For Full Rate Production, DODI 5000.02 requires demonstrated control of the manufacturing processes. The QMS is also critical to the collection of statistical data and demonstration of manufacturing process control.  The philosophy of continuous improvement is essential to a successful production phase and a necessity for meeting cost targets and cost reduction. Data from the QMS drives continuous improvement during the production phase by focusing improvement efforts on high cost drivers and production processes with low yields.  So an effective QMS is a critical element in Full Rate Production.  \n\n\nSources of Information: \nAt this stage in the acquisition process data from production units should be available.  Post Milestone C assessment(s) of system performance, including an MRL Assessment leading into Full Rate Production (FRP), is a primary source of program specific information at this phase.  The manufacturing and quality systems should be defined and proven coming out of the FRP decision, allowing for ongoing efforts to continuously improve quality.\n\n\nExamples of ‘Objective Evidence’:\n•Quality targets verified on the FRP line \n•Evidence that continuous quality improvement is on-going \n•Quality Plan and a Quality Management System documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have the previously established quality targets for Full Rate Production (FRP) been verified by actual quality data?\n2. Is continuous quality improvement on-going?\n3. Are statistical controls being applied where appropriate?\n\n\n\nAdditional Considerations:\n•What quality targets are in place, and what is your status in achieving these targets?\n•Does quality data from Rate Production units provide confidence in the QMS which will be used on this program?\n•Is there enough flexibility in the QMS to accommodate fluctuating production rates (accelerated production or program delay/stretched out production)?\n•Is the QMS compliant with AS9100 or another industry standard? \n\n\nLessons Learned: \n• Following the Full Rate Production decision, there is likely to be a tendency to lose focus on continuous improvement.  There may also be an experience drain as management is moved onto more high risk programs.  Continue to schedule regular reviews of quality metrics and emphasize progress to quality system improvement initiatives to ensure the program’s quality doesn’t regress coming out of Full Rate Production.  Educate any new management entering the program at this phase on the importance of continued focus on quality and continuous improvement.\n• The opportunities to reduce cost late in production are often accompanied with more quality risks than management recognizes.  Make sure the impact on quality is addressed when considering production changes during this phase."]
["$A$22",254]
["$B$22",255,"F.2 - Product Quality"]
["$C$22",256,"Quality metrology state of the art surveyed. Hypotheses developed regarding cause-effect relationships between technology variables and quality.","Purpose: \nTo lay the groundwork for ongoing Quality Management System development concurrent with concept exploration, in parallel with the development of a manufacturing capability for concepts.  To document potential effects of quality on new products or manufacturing technologies and any technology development related to quality processes in the concept phase. Hypotheses of cause-effect relationships between technology variables and quality should be addressed.\n\n\nSources of Information: \nThe majority of information for product quality planning may come from historical data and product quality plans from like systems.  For example, if you are pursuing a new concept you may want to look at previous concept development activities. \n\n\nExamples of ‘Objective Evidence’:\n•  Quality metrology state of the art surveyed and documented\n•  Hypotheses regarding cause-effect relationships between technology variables and quality developed and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has the quality metrology state of the art been surveyed?\n2. Have hypotheses been developed regarding cause-effect relationships between technology variables and quality?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$22",257,"Studies performed to test hypotheses regarding cause-effect relationships between technology variables and quality. Elements identified which have a potential impact on quality (i.e., materials, processes, capabilities, limitations). ","Purpose: \nTo lay the groundwork for ongoing Quality Management System development concurrent with system design, in parallel with the development of a manufacturing capability for system concepts.  To document potential effects of quality on new products or manufacturing technologies and any technology development related to quality processes in the concept phase. Cause-effect relationships between technology variables and quality should be addressed.\n\nSources of Information: \nThe majority of information for product quality planning may come from historical data and product quality plans from like systems.  For example, if you are building a new aircraft you may want to look at previous aerospace product quality plans. \n\nExamples of ‘Objective Evidence’:\n•  Studies to test hypotheses regarding cause-effect relationships between technology variables and quality performed and documented\n•  Elements which have a potential impact on quality (i.e., materials, processes, capabilities, limitations) identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have studies been performed to test hypotheses regarding cause-effect relationships between technology variables and quality?\n2. Have elements been identified which have a potential impact on quality (i.e., materials, processes, capabilities, limitations)?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$E$22",258,"System concept elements evaluated for quality using experiments, modeling and simulation. Initial product quality requirements, risks, and issues identified. Inspection technologies identified.","Purpose: \nTo lay the groundwork for ongoing Quality Management System development concurrent with system design, in parallel with the development of a manufacturing capability for the system.  To document potential effects of quality on new products or manufacturing technologies and any technology development related to quality processes in the Technology Development Strategy. Initial product quality requirements, risks, and issues for system concepts and inspection technologies are documented.\n\nSources of Information: \nThe majority of information for product quality planning may come from historical data and product quality plans from like systems.  For example, if you are building a new aircraft you may want to look at previous aerospace product quality plans. \n\nExamples of ‘Objective Evidence’:\n•  System concept elements for quality using experiments, modeling and simulation have been evaluated and documented\n•  Initial product quality requirements, risks, and issues identified and documented\n•  Inspection technologies identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have system concept elements been evaluated for quality using experiments, modeling and simulation?\n2. Have initial product quality requirements, risks, and issues been identified? \n3. Have inspection technologies been identified?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$F$22",259,"Product quality requirements and the inspection and acceptance testing strategy for the preferred materiel solution considered in AoA and documented in the AS. Product quality risk and issue mitigation plans documented in the SEP.","Purpose: \nThe role of manufacturing in acquisition is to influence the design (make sure it is producible) and prepare for production.  To assess manufacturing feasibility with a focus on product quality beginning with the quality strategy.  The early phases of acquisition often provide opportunities for planning.  At this time there may be no product, so it is a time for establishing the product inspection and acceptance testing strategy.  The product quality strategy should be carried forward and become a part of the Acquisition Strategy (AS) and Systems Engineering Plan (SEP).\n\nSources of Information: \nThe majority of information for product quality planning may come from historical data and product quality plans from like systems.  For example, if you are building a new aircraft you may want to look at old aerospace product quality plans. \n\nExamples of ‘Objective Evidence’:\n•  Product quality requirements and the inspection and acceptance testing strategy for the preferred materiel solution identified and documented as part of the Acquisition Strategy \n•  Product inspection and acceptance testing strategy for the preferred materiel solution included and documented in the Systems Engineering Plan (SEP) and contractor System Engineering Management Plan (SEMP)\n•  Product quality risk and issue mitigation plans documented in the SEP\n\nQuestions (MRL Criteria in the form of a question):\n1. Have product quality requirements and the inspection and acceptance testing strategy for the preferred materiel solution been considered in AoA?\n2. Have product quality requirements and the inspection and acceptance testing strategy for the preferred materiel solution been documented in the AS?\n3. Have product quality risk and issue mitigation plans been documented in the SEP?\n\nAdditional Considerations:\n•  Does the strategy appropriately reflect the industry sector of the item developed and being produced?\n•  Does the strategy appropriately reflect the rates/quantities of items estimated to be produced?\n•  Is the strategy a “best practice?”\n\nLessons Learned: \n•  Product quality must be appropriate for the item(s) being produced.  For example, you should not expect to use statistical process control (SPC) on limited production items.  Conversely, any product quality strategy that includes significant numbers of items should include a provision for SPC.  \n•  A tolerancing and dimensioning strategy should reflect the requirements of the end item.  All too often engineers apply strategy of high tolerances just because the item is an aerospace item.  Certainly many dimensional characteristics will need tight tolerances, but many characteristics perhaps could be looser and still provide for the requisite level of quality.\n"]
["$G$22",260,"Roles and responsibilities identified for acceptance test procedures, in-process and final inspections, and statistical process controls for prototype units. ","Purpose: \nTo influence the design (make sure it is producible) and prepare for production.  To evaluate manufacturing processes.  This includes the evaluation of product quality. Technology development results in the building of prototype components.  It is logical to assume that the roles and responsibilities have been identified for acceptance test procedures to include in-process and final inspections.  As the design matures one of the requirements is to identify key characteristics. Those are the product characteristics that are the most important and are the ones that might require the use of statistical process controls (SPC), if applicable.  Product quality strategies and procedures should be carried forward and matured as product development moves from the component level to the system level.\n\n\nSources of Information: \nThe majority of information will come from QA inspection and acceptance data and from design data.  \n\n\nExamples of ‘Objective Evidence’:\n•  Roles and responsibilities documented for acceptance test procedures, in-process and final inspections \n•  Statistical process controls been identified and documented for prototype units \n•  Policies/procedures documented for identification and management of key characteristics \n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have roles and responsibilities for acceptance test procedures been identified?\n2.  Have roles and responsibilities in-process and final inspections been identified?\n3.  Have roles and responsibilities for statistical process controls for prototype units been identified?\n\n\nAdditional Considerations:\n•  Were the prototype components built and inspected in a production relevant environment?\n•  What constitutes a relevant environment for your product (system)?\n\n\nLessons Learned: \n•  A production relevant environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items.  For example, a production relevant environment for a satellite is different than that used in building prototype air vehicles.\n•  Statistical Process Control is not a requirement for all programs.  Rates and quantities play a major role in the decision to manage product quality using SPC.\n•  Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control.\n"]
["$H$22",261,"KC management approach defined.  Initial requirements identified for acceptance test procedures and in-process and final inspection requirements for EMD units.  Appropriate inspection and acceptance test procedures identified for prototype units.  ","Purpose: \nTo influence the design (make sure it is producible) and prepare for production.  To identify any remaining design, integration and manufacturing risks. The program is ready to exit the TD phase when: (1) The technology and manufacturing processes for that program have been assessed and demonstrated in a relevant environment, and (2) all manufacturing risks have been identified. Technology development results in the building of systems and the integration of subsystems and components.  Roles and responsibilities should have been identified for acceptance test procedures to include in-process and final inspections at the system level.  Key characteristics should have been identified.  Those product characteristics are the most important and might require the use of statistical process controls (SPC).  Product quality strategies and procedures developed in TD should be carried forward and matured as product development moves into EMD.\n\n\nSources of Information: \nThe majority of information will come from QA inspection and acceptance data and design data.   \n\n\nExamples of ‘Objective Evidence’:\n•  Key Characteristic management policy/procedure documented\n•  Acceptance test procedures and in-process and final inspections documentation\n   \n\nQuestions (MRL Criteria in the form of a question):\n1.  Has an approach for management of Key Characteristics been defined?  \n2.  Have initial acceptance test requirements and procedures for EMD units been identified?\n3.  Have roles and responsibilities in-process and final inspections been identified?\n4.  Have appropriate inspection and acceptance test procedures been identified for prototype units?\n\n\nAdditional Considerations:\n•  Have product quality risks been identified?\n•  Is the risk management plan for those risks acceptable (within the scope of the current contract)?\n•  Were the prototype systems built and inspected in a production relevant environment?\n•  What constitutes a relevant environment for your product (system)?\n\n\nLessons Learned: \n•  A production relevant environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items.  For example, a production relevant environment for a satellite is different than that used in building prototype air vehicles.\n•  Statistical Process Control is not a requirement for all programs.  Rates and quantities play a major role in the decision to manage product quality using SPC.\n•  Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control.  \n•  There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n"]
["$I$22",262,"Quality data from the production representative environment collected and analyzed and results used to shape improvement plans. Control plans completed for management of KCs. Test and Inspection plans being developed for EMD units. ","Purpose: \nTo identify and mitigate technical risks.  To collect quality data in a production representative environment and use the results to shape improvement plans.  Since CDR should have been completed, 80-90% of the engineering drawings should have been released. The design or product baseline is close to being frozen.  The contractor’s ability to influence the design is extremely limited at this point as are opportunities to improve performance or reduce costs. A successful CDR will assess and assure that the critical manufacturing processes (including quality) are mature.  Engineering and Manufacturing Development (EMD) results in the completion of the design, or at least, the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality.\n\n\nSources of Information: \nThe majority of information will come from QA inspection and acceptance data and design data.  \n\n\nExamples of ‘Objective Evidence’:\n•Quality data from the production representative environment collected \n•Quality data analyzed and the results used to shape improvement plans\n•Key Characteristic control plans completed\n•Test and Inspection Plans developed for EMD units   \n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has quality data from the production representative environment been collected and analyzed? \n2. Have the results of the analysis of production representative quality data been used to shape improvement plans? \n3. Are control plans for management of Key Characteristics (KCs) complete?\n4. Has development of Test and Inspection Plans for EMD units been initiated?\n\n\nAdditional Considerations:\n•Have product quality targets been established?\n•Do product quality program requirements meet industry standards (control of non-conforming material, corrective action, product handling and storage, inspection records, etc.)?\n\n\nLessons Learned: \n•A production representative environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items.  For example, a production relevant environment for a satellite is different than that used in building prototype air vehicles.\n•Statistical Process Control is not a requirement for all programs.  Rates and quantities play a major role in the decision to manage product quality using SPC.\n•Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control.  \n•There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n•Manufacturing and QA personnel should be involved in the CDR and in any follow-up activities.\n"]
["$J$22",263,"KCs managed. Measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.). Pilot line data meets capability requirements for all KCs. Test and Inspection plans complete and validated for production units.","Purpose: \nTo reduce system-level risk. To integrate components and subsystems, and complete the detailed design to meet performance requirements with a producible and sustainable design. Engineering and Manufacturing Development (EMD) typically includes the demonstration of production prototype articles or engineering development models. EMD results in the completion of the design, at least the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality.\n\n\nSources of Information: \nThe majority of information will come from QA inspection and acceptance data and from factory floor data gathered from the production of prototype articles and from design data from the results of testing.   \n\n\nExamples of ‘Objective Evidence’:\n•Key Characteristics managed with measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)\n•Pilot line data meets capability requirements for all Key Characteristics \n•Test and Inspection Plans complete and validated for production units   \n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are Key Characteristics being actively managed? \n2. Are measurement procedures and controls in place, such as Statistical Process Controls; a Failure, Reporting, Analysis, and Corrective Action System, quality audits, customer satisfaction monitoring, etc.? \n3. Are process capability requirements for all Key Characteristics being met by pilot line data?\n4. Are all Test and Inspection Plans complete?\n5. Have all Test and Inspection Plans for initial production (LRIP) units been validated?\n\n\n\nAdditional Considerations:\n•Are measurement procedures and controls in place (e.g., SPC, FRACAS, audits, customer satisfaction, etc.)?\n•Non-conforming material is properly identified and corrective actions taken.\n•Inspection status is clearly identified.\n•Product quality records are available and meet appropriate requirements.\n•Have quality risks and adequate metrics been identified?\n•Have quality targets been demonstrated on the pilot line?\n•Is continuous quality improvement on-going?\n\n\nLessons Learned: \n•A pilot line environment is different for different products which is often dependent on rates and quantities of items being produced and the complexity of those items.  For example, a pilot line environment for a satellite is different than that used in building prototype air vehicles.\n•Statistical Process Control is not a requirement for all programs.  Rates and quantities play a major role in the decision to manage product quality using SPC.\n•There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n•Some programs are so low volume that their “Full Rate Production” may be only one or two items (e.g., space programs) in which case, the pilot line becomes, in essence, the FRP line. Also, because of low-rate production, there may not be sufficient data to do a formal yield plot for key characteristics.\n•Even though some programs are low volume (e.g., satellites or certain aircraft) there may be individual manufacturing processes used in production that are repeated sufficiently to gather yield date (e.g., manufacture and installation of solar cells, riveting of fuselage skins, soldering of wiring harnesses to connectors, etc.)\n•FRACAS reports can be a valuable source of information indicating potential process issues (inconsistent or variable processes impact product reliability).\n•Look for Pareto charts of major defects uncovered during inspection to identify process problems.  \n"]
["$K$22",264,"Data from LRIP demonstrates production processes, for all KCs and other manufacturing processes critical to quality, are capable and under control for FRP.","Purpose: \nTo provide an adequate and efficient manufacturing capability. To achieve operational capability to satisfy mission needs.  This effort should result in completion of manufacturing development to ensure capability to produce the minimum quantity of production-representative articles for IOT&E, establish an initial production base for the system, and permit orderly increase in production rate leading to full-rate production when IOT&E is successfully completed.\n\n\nSources of Information: \nThe majority of information will come from QA inspection and acceptance data and from production data.    \n\n\nExamples of ‘Objective Evidence’:\n•Key Characteristic management policy/procedure documented\n•Data from LRIP demonstrate that production processes for all Key Characteristics and other manufacturing processes critical to quality are capable and under control for FRP \n•Acceptance test procedures and in-process and final inspections documentation   \n\n\nQuestions (MRL Criteria in the form of a question):\n1. Does analysis of Low Rate Initial Production (LRIP) quality data show that all Key Characteristics, as well as other critical-to-quality manufacturing processes are capable and under control for FRP?\n\nAdditional Considerations:\n• Are measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Non-conforming material is properly identified and corrective actions taken.\n• Inspection status is clearly identified.\n• Product quality records are available and meet appropriate requirements.\n\n\nLessons Learned: \n• A low-rate initial production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items.  For example, a pilot line environment for a satellite is different than that used in building prototype air vehicles.\n• Statistical Process Control is not a requirement for all programs.  Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
["$L$22",265,"KCs controlled at rate. Results achieve targeted statistical level on all KCs. Results reflect continuous improvement.","Purpose: \nTo provide operational items to the warfighter and achieve an operational capability that satisfies mission needs. The knowledge required to support system approval must include demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes.\n\n\nSources of Information: \nThe majority of information will come from QA inspection and acceptance data and from production data.    \n\n\nExamples of ‘Objective Evidence’:\n•Key Characteristic management control documented\n•Quality data documentation provides evidence that targeted statistical level on all Key Characteristics have been achieved  \n•Quality data reflects continuous improvement\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are all Key Characteristics being managed and in control for FRP?\n2. Are all Key Characteristics meeting required targets?\n3. Does quality data show continuous improvement?\n\n\nAdditional Considerations:\n• Are measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Non-conforming material is properly identified and corrective actions taken.\n• Inspection status is clearly identified.\n• Product quality records are available and meet appropriate requirements.\n• Is the contractor conducting continuous improvement activities?\n\n\nLessons Learned: \n• A full-rate production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items\n• Statistical Process Control is not a requirement for all programs.  Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
["$A$23",266]
["$B$23",267,"F.3 - Supplier Quality Management"]
["$C$23",268,"Supplier quality and quality management systems state of the art surveyed.","Purpose: \nTo lay the groundwork for ongoing Quality Management System development concurrent with system design, in parallel with the development of a manufacturing capability for system concepts as they apply to supplier quality management.  To document potential effects of quality on new supplier products or manufacturing technologies and any technology development related to quality processes in the concept phase.  \n\n\nSources of Information:  \nThis data should come from supplier management, quality, design, and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:                                  \n•  Supplier quality and quality management systems state of the art surveyed and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have supplier quality and quality management systems state of the art been surveyed?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$23",269,"Initial supplier quality and quality management systems evaluated.","Purpose: \nTo lay the groundwork for ongoing Quality Management System development concurrent with system design, in parallel with the development of a manufacturing capability for system concepts as they apply to supplier quality management.  To document potential effects of quality on new supplier products or manufacturing technologies and any technology development related to quality processes in the concept phase.  \n\nSources of Information:  \nThis data should come from supplier management, quality, design, and manufacturing departments.\n\nExamples of ‘Objective Evidence’:                                  \n•  Initial supplier quality and quality management systems evaluated and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have initial supplier quality and quality management systems been evaluated?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$E$23",270,"Supplier quality and quality management system requirements for system concepts identified.","Purpose: \nTo lay the groundwork for ongoing Quality Management System development concurrent with system design, in parallel with the development of a manufacturing capability for the system.  To document potential effects of quality on new products or manufacturing technologies and any technology development related to quality processes in the Technology Development Strategy. Supplier quality and quality management system requirements for the system concept have been identified.  \n\nSources of Information:  \nThis data should come from supplier management, quality, design, and manufacturing departments.\n\nExamples of ‘Objective Evidence’:                                  \n•  Supplier quality and quality management system requirements for the system concept have been identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have supplier quality and quality management system requirements for the system concept been identified?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$F$23",271,"Potential supplier quality capabilities, risks, and issues identified for the preferred materiel solution, including subtier suppliers. Supplier quality management system requirements defined, and documented in the AS. ","Purpose:  \nTo begin assessing the quality capability of potential suppliers and to understand the supplier quality issues/risk in the AoAs. To use this information to understand the risk in selecting the alternatives to proceed with into TMRR. To use this information to quantify the uncertainty/risk of the achieving the program objectives. The program manager must understand the capability of the supplier base to provide quality products for each alternative design being considered and reflect risk in program planning efforts. It is important to survey potential suppliers’ quality systems capability to meet program requirements and identify risk presented by each supplier.  At the end of this phase you will select the best alternatives available to address the user’s requirements to proceed into TMRR.  \n\n\nSources of Information:  \nThis data should come from supplier management, quality, design, and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Documentation for potential supply base quality capabilities and risks identified, to include subtier quality management \n•   AS includes defined and documented supplier quality management system requirements\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have potential supplier quality capabilities, risks, and issues been identified for the preferred materiel solution, including subtier suppliers?\n2. Have supplier quality management system requirements been defined and documented in the AS? \n\n\nAdditional Considerations:  \n• Do the suppliers’ quality systems appropriately reflect the industry sector of the item developed and being produced?\n• Does the suppliers’ quality strategy appropriately reflect the rates/quantities of items estimated to be produced?\n• Is the supplier quality strategy a “best practice?”\n\n\nLessons Learned: \n• Supplier product quality must be appropriate for the item(s) being produced.  For example, you should not expect to use statistical process control (SPC) on limited production items.  Conversely, any product quality strategy that includes significant numbers of items should include a provision for SPC.  \n• A supplier tolerancing and dimensioning strategy should reflect the requirements of the end item.  All too often engineers apply strategy of high tolerances just because the item is an aerospace item.  Certainly many dimensional characteristics will need tight tolerances, but many characteristics perhaps could be looser and still provide for the requisite level of quality.\n"]
["$G$23",272,"Supply base quality capabilities and risks identified, including subtier supplier quality management.","Purpose:  \nTo assess the quality capability of suppliers producing prototype items and identify supplier quality issues/risk. To use this information to quantify the uncertainty/risk of the achieving the program objectives and implement mitigation plans to reduce that risk. The program manager must assess supplier quality capabilities against program requirements and institute plans to achieve the quality requirements.  \n\n\nSources of Information:  \nThis data should come from supplier management, quality, design, and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Documentation for supply base quality capabilities and risks, to include subtier quality management \n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have quality capabilities and risks from the supply chain (i.e., supplier base) been identified? \n2.  Were the quality capabilities and risks from subtier supplier quality management also identified?\n\n\nAdditional Considerations:  \n•  Were the supplier prototype items built and inspected in a production relevant environment?\n•  What constitutes a relevant environment for your supplier?\n\n\nLessons Learned: \n•  A production relevant environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items.  Supplier quality systems must be validated to ensure they are using production relevant processes.\n•  Many suppliers do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control. \n•  Programs have been shut down because of supplier QA issues.\n"]
["$H$23",273,"Supply base quality improvement initiatives identified addressing supplier QMS shortfalls, including subtier supplier quality management. ","Purpose:  \nTo address supplier quality system issues/concerns building prototype items in a production relevant environment. To use this data to make improvement plans to achieve future requirements.  The program needs to understand suppliers’ quality systems and capabilities vs. requirements. All issues/concerns need to have mitigation plans in place.  If this is not addressed before entering EMD, it could become a major problem downstream. \n\n\nSources of Information:  \nThis data should come from suppler management, quality, design, and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Supply base quality improvement initiatives documentation\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have initiatives to address shortfalls in supplier Quality Management Systems been identified?\n2.  Have initiatives to address shortfalls in supply chain Quality Management been identified?\n\n\nAdditional Considerations:  \n•  Have supplier product quality risks been identified?\n•  Is the risk management plan for supplier risks acceptable (within the scope of the current contract)?\n•  Were supplier prototype systems built and inspected in a production relevant environment?\n•  What constitutes a relevant environment for your supplier?\n\n\nLessons Learned: \n•  A production relevant environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items.  Supplier quality systems must be validated to ensure they are using production relevant processes.\n•  Statistical Process Control is not a requirement for all suppliers.  Rates and quantities play a major role in the decision to manage product quality using SPC.\n•  Many suppliers do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control.  \n•  There should be a clear path of requirements flow down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate. \n•  Per Quality Management Systems requirements (e.g., AS9100), the Quality Manager is to report to the ‘Executive Level’ within companies.\n"]
["$I$23",274,"Key supplier QMSs meet appropriate industry standards. Supplier quality data from production representative units collected and analyzed. Strategy for audits of critical supplier processes outlined. ","Purpose:  \nTo focus on key suppliers and ensure they are using appropriate QMS. To use the data generated from building units in a production representative environment to compare capability versus program requirements. To ensure prime has a strategy in place to assess critical suppliers’ in-place. The program manager must consistently monitor the performance and capability of key/major suppliers. \n\n\nSources of Information:  \nThis data should come from suppler management, quality, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Supplier Quality Management Systems meet appropriate industry standards \n•Supplier quality data from production representative units has been collected and analyzed \n•Strategy outlined for audits of critical supplier processes\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Do key supplier Quality Management Systems (QMSs) meet appropriate industry standards?\n2. Has supplier quality data from production representative units been collected and analyzed?\n3. Has a strategy for audits of critical supplier processes been outlined?\n\n\n\nAdditional Considerations:  \n•Have supplier product quality targets been established?\n•Do supplier quality program requirements meet industry standards (control of non-conforming material, corrective action, product handling and storage, inspection records, etc.)?\n\n\nLessons Learned: \n•A production representative environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items.  Supplier quality systems must be validated to ensure they are using production representative processes.\n•Statistical Process Control is not a requirement for all suppliers.  Rates and quantities play a major role in the decision to manage product quality using SPC.\n•Many suppliers do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control.  \n•There should be a clear path of requirements flow-down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n•We need to continually assess the systems and capabilities of our suppliers as we progress into production. Historically a large percent of the cost and problems have come from suppliers and being informed in this area is critical in achieving program objectives. At this point the Prime contractor should be aggressively engaging with their major and critical subcontractors and vendors where known or potential risks may reside.  Subcontractor QA is now a contact sport.\n•The prime contractor needs a system in place to ensure that appropriate quality requirements get flowed down to suppliers through the entire Work Breakdown Structure (WBS). \n•Per Quality Management Systems requirements (e.g., AS9100), the Quality Manager is to report to the ‘Executive Level’ within companies.\n"]
["$J$23",275,"Supplier program-specific QMSs adequate. Supplier products qualification testing and first article inspection completed. Acceptance testing of supplier products adequate to begin LRIP. Plan for subcontractor process audits in place and implemented by prime contractor.","Purpose:  \nTo reduce system-level risk. Through the conduct of systems engineering, the efforts in this phase integrate components and subsystems and complete the detailed design to meet performance requirements with a producible and sustainable design. EMD typically includes the demonstration of production prototype articles or engineering development models. Major design issues are well within program constraints and have limited impact on product quality. \n\n\nSources of Information:  \nThe majority of information will come from vendor/supplier QA inspection and acceptance data and from factory floor data gathered from the production of prototype articles and from design data from the results of testing. Materiel managers at the prime can also provide information on supplier product quality.\n\n\nExamples of ‘Objective Evidence’:\n•Supplier program-specific Quality Management Systems documented\n•Supplier products have completed qualification testing \n•Supplier products have completed first article inspection (FAI) \n•Acceptance testing of supplier products documented and adequate to begin LRIP \n•Plan for subcontractor process audits in place and being implemented by prime contractor\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are supplier program-specific Quality Management Systems (QMSs) adequate (i.e., meet all quality requirements)?\n2. Is qualification testing for all products from suppliers completed?\n3. Are all First Article Inspections (FAIs) for all products from suppliers completed?\n4. Is acceptance testing of supplier products (i.e., processes and throughput) adequate to begin initial production (LRIP)?\n5. Are plans to conduct quality audits of subcontractor processes in place? \n6. Have quality audits of subcontractor processes been implemented?\n\n\nAdditional Considerations:  \n•What supplier quality targets are in place, and what is your status in achieving these targets?\n•Does quality data from supplier production representative and pilot line units provide confidence in the QMS which will be used on this program?\n•Are measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n•Is supplier non-conforming material properly identified and corrective actions taken?\n•Are supplier product quality records are available and meet appropriate requirements?\n•Have quality risks and adequate metrics been identified?\n•Have quality targets been demonstrated on the pilot line?\n•Is continuous quality improvement on-going at the supplier?\n\n\nLessons Learned: \n•A pilot line environment is different for different products which is often dependent on rates and quantities of items being produced and the complexity of those items.  Supplier quality systems must be validated to ensure they are using production processes.\n•Statistical Process Control is not a requirement for all programs.  Rates and quantities play a major role in the decision to manage product quality using SPC.\n•There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n•Even though some programs are low volume (e.g., satellites or certain aircraft) there may be individual manufacturing processes used in production that are repeated sufficiently to gather yield date (e.g., manufacture and installation of solar cells, riveting of fuselage skins, soldering of wiring harnesses to connectors, etc.)\n•Supplier FRACAS reports can be a valuable source of information indicating potential process issues (inconsistent or variable processes impact product reliability).\n•Look for supplier Pareto charts of major defects uncovered during inspection to identify process problems.\n"]
["$K$23",276,"Supplier quality management of KCs and other critical manufacturing processes demonstrates capability and control for FRP.  Acceptance testing of supplier products reflects control of quality adequate to begin FRP.  Subcontractor quality audits performed as necessary to ensure subcontractor specification compliance.","Purpose:  \nTo use demonstrated results from a LRIP line to verify adequate supplier controls and capability exists to meet FRP requirements. To ensure that continued audits and reviews are performed on suppliers as appropriate. The program manager must verify capability and controls from suppliers by using actual build data from LRIP. This early production data will lay the foundation for future production efforts.  Management emphasis must be placed at suppliers where necessary to ensure production objectives can be achieved. \n\n\nSources of Information:  \nThis data should come from supplier management, quality, design and manufacturing departments.\n \n\nExamples of ‘Objective Evidence’:\n•Evidence of supplier management of quality of Key Characteristics and other critical manufacturing processes demonstrate capability and control for FRP\n•Evidence of supplier management of quality of Key Characteristics and other critical manufacturing processes demonstrate capability and control for FRP\n•Subcontractor Quality Audits documentation\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Does analysis of supplier quality data show that supplier quality management of Key Characteristics, as well as other critical-to-quality manufacturing processes are capable and under control ready for rate production (FRP)? \n2. Is acceptance testing of supplier products (i.e., processes and throughput) adequate to begin FRP?\n3.  Are quality audits of subcontractor processes to ensure subcontractor compliance to specifications being performed?\n\n\n\nAdditional Considerations:  \n• Are supplier measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Is supplier non-conforming material is properly identified and corrective actions taken?\n• Supplier inspection status is clearly identified.\n• Supplier product quality records are available and meet appropriate requirements.\n\n\nLessons Learned: \n• A low-rate initial production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items.  Supplier quality systems must be validated to ensure they are using production processes.\n• Statistical Process Control is not a requirement for all programs.  Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
["$L$23",277,"Supplier quality data reflects adequate management of KCs and control of critical manufacturing processes, including quality management down to subtier suppliers.  Results achieve high statistical level (e.g., 6-sigma) on all critical dimensions.  Subcontractor quality audits performed as necessary to ensure subcontractor specification compliance.","Purpose:  \nTo verify the supplier base has adequate controls and capability to achieve production rate requirements.  The program manager must constantly verify the supplier base controls and capability by using actual build data from supplier production lines to ensure confidence in achieving production requirements.\n\n\nSources of Information:  \nThis data should come from supplier management, quality, design and manufacturing departments.\n \n\nExamples of ‘Objective Evidence’:\n•Supplier quality data reflects adequate management of Key Characteristics and control of critical manufacturing processes\n•Process quality data shows evidence of high statistical level (e.g. 6-sigma) on all critical dimensions \n•Subcontractor Quality Audits documentation\n•Supply base quality improvement initiatives documentation\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Does analysis of supplier quality data show adequate management of Key Characteristics and control of critical manufacturing processes  (i.e., meeting Cpks), including supplier quality management of subtier suppliers?\n2. Do results of analysis of supplier quality data show a high statistical level (e.g. 6-sigma) on all critical dimensions?\n3. Are quality audits of subcontractor processes to ensure subcontractor compliance to specifications being performed?\n\n\n\n\nAdditional Considerations:  \n• Are supplier measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Supplier non-conforming material is properly identified and corrective actions taken.\n• Supplier inspection status is clearly identified.\n• Supplier product quality records are available and meet appropriate requirements.\n• Is the supplier conducting continuous improvement activities?\n\n\nLessons Learned: \n• Historically a large percent of the cost and problems have come from suppliers not staying up to speed in this area. The supplier base will be a good area to look for cost reductions. \n• Many prime contractors and their subcontractors will focus their efforts on continuous process improvement if it is a priority of the program office.\n• A full-rate production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items\n• There should be a clear path of requirements flow-down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
["$A$24",278,"G - Mfg Workforce (Engineering & Production)","Manufacturing Workforce (Engineering and Production): Requires an assessment of the required skills, availability, and required number of personnel to support the manufacturing effort.\n\nManufacturing Management includes manufacturing planning and scheduling, materials planning, and tooling/special test and inspection equipment."]
["$B$24",279,"G.1 - Mfg Workforce (Engineering & Production)","The loss or impending loss of manufacturers of items or suppliers of items or the loss or impending loss of raw materials used to manufacture items of supply. Such losses may cause material shortages that endanger a weapon system’s or equipment’s development, production, or post-production support capability.  (DoDM 4140.01, Volume 3)"]
["$C$24",280,"Workforce skill sets to support emerging trends in manufacturing and technology surveyed.","Purpose: \nTo evaluate whether new manufacturing skills will be needed to produce the alternatives being considered and if so, to assess the risk of obtaining required skills to build the prototypes needed. Contractors need to establish very early whether new skills will be required and risk associated with obtaining those skills to meet program requirements. The skills might have to be developed prior to building prototype articles in the earliest phase of a program. \n\n\nSources of Information: \nThe Quality Manager, Design Engineer, Manufacturing Engineer, and Resource Management will be good places to get an understanding of the skills needed. \n\n\nExamples of ‘Objective Evidence’:\n•  Workforce skill sets to support emerging trends in manufacturing and technology surveyed and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have workforce skill sets to support emerging trends in manufacturing and technology been surveyed?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$24",281,"Workforce skill sets to support emerging trends in manufacturing and technology evaluated.","Purpose: \nTo evaluate whether new manufacturing skills will be needed to produce the alternatives being considered and if so, to assess the risk of obtaining required skills to build the products needed. Contractors need to establish very early whether new skills will be required and risk associated with obtaining those skills to meet program requirements. The skills might have to be developed prior to building test units in the earliest phase of a program, possibly before Milestone A. \n\nSources of Information: \nThe Quality Manager, Design Engineer, Manufacturing Engineer, and Resource Management will be good places to get an understanding of the skills available at their facility. \n\nExamples of ‘Objective Evidence’:\n•  Workforce skill sets to support emerging trends in manufacturing and technology evaluated and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have workforce skill sets to support emerging trends in manufacturing and technology been evaluated?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$E$24",282,"Workforce skill set requirements for system concepts identified. Workforce skill set capability gaps identified","Purpose: \nTo evaluate whether new manufacturing skills will be needed to produce the alternatives being considered and if so, to assess the risk of obtaining required skills to build the products needed. Contractors need to establish very early whether new skills will be required and risk associated with obtaining those skills to meet program requirements. The skills might have to be developed prior to building test units in the earliest phase of a program, possibly before Milestone A. \n\nSources of Information: \nThe Quality Manager, Design Engineer, Manufacturing Engineer, and Resource Management will be good places to get an understanding of the skills available at their facility.\n\nExamples of ‘Objective Evidence’:\n•  Workforce skill set requirements for system concepts identified and documented\n•  Workforce skill set capability gaps identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have workforce skill set requirements for system concepts been identified?\n2. Have workforce skill set capability gaps been identified?\n\nAdditional Considerations:\n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?\n\nLessons Learned: \n• When you are looking at skills, you must account for all the resources required to build, test/inspect and support (e.g. Manufacturing Engineers) the units. "]
["$F$24",283,"Workforce skill set and production workforce requirements (technical and operational) for the preferred materiel solution identified and considered in the AoA. Workforce training and development requirements to close skill set gaps defined. Availability of workforce for TMRR Phase determined.","Purpose:\nTo perform a top level assessment for the program looking for areas of risk driven by workforce skills requirements. Where risks are identified, appropriate risk reduction efforts must be put into place in the TMRR Phase.  In addition, cost should be reflected in both the estimates to execute the TMRR phase and overall lifecycle cost projections. The program must ensure the manufacturing workforce risk is assessed for each proposed alternative to make informed decisions on total risk of achieving program cost, schedule, and performance objectives.\n\n\nSources of Information: \nThe Quality Manager, Design Engineer, Manufacturing Engineer, and Resource Management will be good places to get an understanding of the skills available at their facility.\n\n\nExamples of ‘Objective Evidence’:\n•  Analysis of Alternatives (AoA)\n•  Necessary manufacturing workforce skills identified and documented – employee & personnel training records\n•  Production workforce requirements (technical and operational) evaluated – employee & personnel training records\n•  Availability of the process development workforce for the Technology Maturation and Risk Reduction Phase documented\n\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have workforce skill set and production workforce requirements (technical and operational) for the preferred materiel solution been identified?\n2. Have workforce skill set and production workforce requirements (technical and operational) for the preferred materiel solution been considered in the AoA?\n3. Have workforce training and development requirements to close skill set gaps been defined?\n4. Has the availability of the workforce for TMRR Phase been determined?\n\n\n\nAdditional Considerations:\n•  Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?\n\n\nLessons Learned: \n•  When you are looking at skills, you must account for all resources required to build, test/inspect and support labor (e.g. Manufacturing Engineers) for the units. \n•  If there is a new process, or tolerances are more difficult, there exists a stronger probability of new or critical skills required that may be difficult to obtain.\n"]
["$G$24",284,"Skill sets identified and plans developed to meet prototype and production needs. Special skills certification and training requirements established.  ","Purpose: \nTo reduce risk by producing prototype components in a production relevant environment by monitoring and assessing the workforce skills needed to produce prototypes and eventually production units. To begin identifying special skills needed for building prototype hardware and what special training, if any, is required. This effort will be used to ensure resources are available to produce prototype units in a relevant environment.  The top level review can also be used to determine needed skills for future production and cost requirements.\n\n\nSources of Information: \nThe majority of the information regarding prototype build in a production relevant environment will come from the manufacturing/industrial engineering and quality departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Necessary manufacturing workforce skills identified and documented – employee & personnel training records\n•  Manufacturing workforce skill sets plan for prototype production – employee & personnel training records\n•  Special skills certification and training requirements established and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have the required skill sets to meet prototype and production been identified?\n2.  Are plans to meet prototype and production workforce requirements developed?\n3.  Have requirements for special skills certification and training (i.e., included in workforce plan(s) been established? \n\n\nAdditional Considerations:\n•  Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?\n\n\nLessons Learned: \n•  Focus on skills required for new processes, materials or performance requirements either in building or inspection of the products. \n•  If new or critical skills are required to produce the prototype units, be sure you have a good understanding of contractor plans for obtaining these resources.  This area must be tracked closely because it is typically ignored, yet it requires the same level of attention as other critical areas.\n"]
["$H$24",285,"Manufacturing workforce skills available for the production relevant environment. Resources (quantities and skill sets) identified and initial plans developed to achieve requirements for pilot line and production. ","Purpose: \nTo evaluate workforce skills by building prototypes in a production relevant environment. To assess and identify workforce skills needed to produce the development units for the EMD and eventually, Production. This is where you must demonstrate special skills to build prototype hardware and any special training required. This effort will be used to plan for resources to produce development units in a Production Representative and Pilot Line. \n\n\nSources of Information: \nThe majority of the information regarding needed skills will come from the Manufacturing/ Industrial Engineering, Quality, and Human Resources Departments. \n \n\nExamples of ‘Objective Evidence’:\n•  Manufacturing workforce skills documented – employee & personnel training records\n•  Manufacturing workforce resource plan – employee & personnel training records\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Are the required manufacturing workforce skills available for the production relevant environment?\n2.  Have the manufacturing workforce resources required to achieve pilot line and production requirements (i.e., number of personnel and skill sets) been identified? \n3.  Have the plans to achieve the manufacturing workforce resources required to achieve pilot line and production requirements been developed? \n\n\nAdditional Considerations:\n•  Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?\n\n\nLessons Learned: \n•  You must work with the contractor/ supplier to identify what skills are required on a pilot line for your program. \n•  Examine work effort that will migrate from labs and prototype areas to the factory floor, specifically new skills needed versus what are available.\n•  Ensure plans address the skills needed by factory floor personnel are adequate to demonstrate a pilot line in the next phase. \n•  One problem seen in entering EMD is that you may have been able to produce development units with highly skilled technicians, but your manpower skills are insufficient in a production environment.\n"]
["$I$24",286,"Manufacturing workforce resource requirements identified and plans developed to achieve pilot line requirements.  Plans to achieve LRIP workforce requirements updated.  Pilot line workforce trained in production representative environment.","Purpose: \nTo understand the risk of obtaining workforce skills to meet production requirements. To understand the magnitude of this risk, you must build development units in a production representative environment.  This requires monitoring, assessing and identifying workforce skills needed to produce the development units for EMD and training the workforce for pilot line, and eventually, LRIP. Workforce skills requirements addressed during CDR as they relate to design requirements should be and reflected in cost and schedule projections for the Production phase. Here you will begin demonstrating special skills needed to build development unit hardware and what special training, if any, is needed for a production representative environment. This effort will be used to plan for resources required to produce development units in a Pilot Line. \n\n\nSources of Information: \nThe majority of the information, regarding skills needed, will come from the Manufacturing/ Industrial Engineering, Quality, and Human Resources Departments.   \n\n\nExamples of ‘Objective Evidence’:\n•Manufacturing workforce resource requirements identified for the pilot line \n•Plans developed to achieve pilot line requirements \n•Plans updated to achieve Low Rate Initial Production (LRIP) workforce requirements\n•Pilot line workforce trained in the production representative environment – employee & personnel training records\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have the manufacturing workforce resources required to achieve pilot line requirements (i.e., number of personnel and skill sets) been identified?\n2. Have the plans to achieve the manufacturing workforce resources required to achieve pilot line requirements been developed? \n3. Have the plans to achieve the manufacturing workforce resources required to achieve initial production (LRIP) requirements been updated?\n4. Has the pilot line workforce been trained in the production representative environment?\n\n\n\nAdditional Considerations:\n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?\n• Does personnel turnover and/or retention affect the ability to have adequate capabilities available?\n• Is new personnel training adequate to provide the necessary skills?\n\nLessons Learned: \n• You must work with the contractor/ supplier to identify what represents a pilot line for your program. \n• A key area to look at in this phase is work effort that will migrate from labs and prototype areas to the factory floor from TMRR to EMD, specifically what new skills are needed versus what are available.\n• Ensure plans that address the skills needed by factory floor personnel are adequate to demonstrate a pilot line. \n• Retirement of an aging workforce can affect the availability of critical skills.\n• You must place special emphasis on getting factory personnel to produce the EMD hardware as much as possible and understand the risk as you migrate to production. If unique skills are required in production, ensure they are reflected in cost and schedule risk assessments. This early phase will focus more on the planning effort to migrate to the factory floor.\n"]
["$J$24",287,"Manufacturing workforce resource requirements identified and plans developed to achieve LRIP requirements.   LRIP personnel trained on pilot line where possible. Plans to achieve FRP workforce requirements initiated based on pilot line. \n","Purpose: \nTo demonstrate workforce skills required for production by producing development units in a pilot line environment, and to ensure the workforce has established an affordable and executable manufacturing process.  This is where you demonstrate the skills needed for building production unit hardware. This information will be used to plan resources to make production units. The results of your pilot line will be used to assess the risk of achieving production projections on cost and schedule. \n\n\nSources of Information: \nThe majority of the information regarding needed skills will come from the Manufacturing/ Industrial Engineering, Quality, and Human Resources Departments.  \n\n\nExamples of ‘Objective Evidence’:\n•Manufacturing workforce skills documented – employee & personnel training records\n•Manufacturing workforce resources requirements identified for Low Rate Initial Production (LRIP)\n•Plans developed to achieve LRIP requirements \n•Plans updated to achieve Full Rate Production (FRP)\n•LRIP personnel trained on the pilot line where possible\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have the manufacturing workforce resources required to achieve initial production (LRIP) requirements (i.e., number of personnel and skill sets) been identified?\n2. Have the plans to achieve the manufacturing workforce resources required to achieve initial production (LRIP) requirements been developed?\n3. Have production personnel been trained on the pilot line where possible?\n4. Based on pilot line results, have the plans to achieve the manufacturing workforce resources required for Full Rate Production (FRP) requirements been initiated? \n\n\n\nAdditional Considerations:\n• Place special emphasis on any new skills being required on the factory floor.\n• Does personnel turnover and/or retention affect the ability to have adequate capabilities available?\n• Is new personnel training adequate to provide the necessary skills?\n• If there is no pilot line, it may not be possible to train personnel on that pilot line.\n\nLessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier on what represents a pilot line for your programs. \n• Retirement of an aging workforce can affect the availability of critical skills.\n• You must place special emphasis on getting factory personnel to produce the EMD hardware as much as possible and understand the risk as you migrate to production. Ensure unique skills required in production are reflected in cost and schedule risk assessments.\n\n"]
["$K$24",288,"LRIP personnel requirements met. Plan to achieve FRP workforce requirements implemented. ","Purpose: \nTo demonstrate that manufacturing processes are under control and that workforce skills are available to achieve requirements (performance, schedule and cost objectives).  To assess whether workforce planning has been adequately demonstrated/validated to allow for an orderly increase in production rate.  This is where you will validate that planning has provided for the necessary special skills to ensure your processes are under control for building production units.\n\n\nSources of Information: \nThe majority of the information regarding needed skills will come from the Manufacturing/ Industrial Engineering, Quality and Human Resources Departments.  \n\n\nExamples of ‘Objective Evidence’:\n•Manufacturing workforce skills documented – employee & personnel training records\n•Manufacturing workforce resource plan – employee & personnel training records\n•Evidence that LRIP personnel requirements been met\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have all workforce requirements for Low Rate Initial Production (LRIP) been met? \n2. Have the plans to achieve the manufacturing workforce resources required for rate production (FRP) requirements been implemented?\n\n\n\nAdditional Considerations:\n• Ensure production personnel are properly cross-trained for critical skills to avoid work stoppages in case of absences.\n• Does personnel turnover and/or retention affect the ability to have adequate capabilities available?\n• Is new personnel training adequate to provide the necessary skills?\n\nLessons Learned: \n• As you transition to FRP, examine union agreements and other contingencies that might impact higher rates. \n• Retirement of an aging workforce can affect the availability of critical skills.\n"]
["$L$24",289,"FRP personnel requirements met.  Production workforce skill sets maintained in spite of workforce attrition. ","Purpose: \nTo demonstrate that the manufacturing processes are under control and that workforce skills are available to achieve Full Rate Production requirements (performance, schedule and cost objectives).  To assess whether workforce planning has been adequately demonstrated/validated to allow for a possible surge in production rate.\n\n\nSources of Information: \nThe majority of the information regarding needed skills will come from the Manufacturing/ Industrial Engineering, Quality and Human Resources Departments.  \n \n\nExamples of ‘Objective Evidence’:\n•Manufacturing workforce skills documented – employee & personnel training records\n•Manufacturing workforce resource plan – employee & personnel training records\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have all workforce requirements for FRP been met?\n2. In spite of any workforce attrition, are required production workforce skill sets being maintained?\n\n\n\nAdditional Considerations:\n• Are production workforce skill sets being maintained due to attrition of the workforce? \n• Are workforce personnel available, or can they be trained, to meet surge requirements if necessary?\n• Ensure production personnel are properly cross-trained for critical skills to avoid work stoppages in case of absences.\n\n\nLessons Learned: \t\n• One consideration as you migrate to higher rate production will be to examine union agreements and other contingencies that might impact higher rates.\n• Also look at the personnel demographics for key skills."]
["$A$25",290,"H - Facilities","Facilities: Requires an analysis of the capabilities and capacity of key manufacturing facilities (prime, subcontractor, supplier, vendor, and maintenance/repair)."]
["$B$25",291,"H.1 - Tooling / Special Test and Inspection Equipment (STE/SIE)"]
["$C$25",292,"State of the art tooling, test and inspection equipment surveyed.","Purpose:  \nTo identify tooling, test and inspection equipment for characterizing the state of the manufacturing state of the art through the outcome of scientific discovery. To use this information to understand cost and performance risk of selecting the best alternative and to begin risk planning to address potential risks. The contractor must determine whether new/specialized tooling, STE, and SIE are needed to produce hardware for the conceopts being considered. If special tooling, STE, and SIE will be needed, you must begin addressing the risk to achieving program requirements.\n\n\nSources of Information:  \nThis data should come from facilities, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  State of the art tooling, test and inspection equipment surveyed and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has state of the art tooling, test and inspection equipment been surveyed?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$25",293,"Potential tooling, STE, and SIE requirements identified.","Purpose:  \nTo identify whether new/specialized tooling, STE, and SIE requirements for characterizing the state of the manufacturing state of the art through the outcome of scientific discovery. To use this information to understand cost and performance risk of selecting the best alternative and to begin risk planning to address potential risks. The contractor must determine whether new/specialized tooling, STE, and SIE are needed to produce items being considered. If special tooling, STE, and SIE will be needed, you must begin addressing the risk to achieving program requirements.\n\nSources of Information:  \nThis data should come from facilities, design and manufacturing departments.\n\nExamples of ‘Objective Evidence’:\n•  Potential tooling, STE, and SIE requirements for concepts identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have potential tooling, STE, and SIE requirements been identified?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$E$25",294,"Tooling, STE, and SIE requirements for system concepts identified. Special tooling, STE, and SIE gaps identified.","Purpose:  \nTo identify whether new/specialized tooling, STE, and SIE requirements for system concepts will be required to produce proposed alternatives. To use this information to understand cost and performance risk of selecting the best alternative and to begin risk planning to address identified risks. The contractor must determine whether new/specialized tooling, STE, and SIE are needed to produce items being considered. If special tooling, STE, and SIE will be needed, you must begin addressing the risk to achieving program requirements.\n\nSources of Information:  \nThis data should come from facilities, design and manufacturing departments.\n\nExamples of ‘Objective Evidence’:\n•  Tooling, STE, and SIE requirements for system concepts identified and documented\n•  Special tooling, STE, and SIE gaps identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have tooling, STE, and SIE requirements for system concepts been identified?\n2. Have special tooling, STE, and SIE gaps been identified?  \n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$F$25",295,"Tooling/STE/SIE requirements for the preferred materiel solution considered as part of AoA.","Purpose:  \nTo evaluate whether new/unique tooling and test equipment will be required to produce proposed alternatives and to understand overall requirements of tooling and test equipment as it will impact the cost and performance risk. To begin risk planning. Contractors need to establish very early their tooling and test equipment concepts and whether they must develop new/unique tooling and test equipment to produce the proposed alternatives.  If so, the risk must be addressed to achieving program requirements.\n\n\nSources of Information:  \nThis data should come from both design and manufacturing engineering working together along with test engineering, manufacturing tooling and test departments.\n\n\nExamples of ‘Objective Evidence’:\n• Prototype tooling and STE/SIE requirements been identified and documented as part of AoA\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have tooling/Special Test Equipment (STE)/Special Inspection Equipment (SIE) requirements for the preferred materiel solution been considered as part of AoA?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• Tooling and test equipment can be major cost and risk drivers and must be addressed in the AoA. If new/unique tooling and test equipment is required, this must be tracked through EMD.\n"]
["$G$25",296,"Tooling and STE/SIE requirements identified with supporting rationale and schedule. ","Purpose:  \nTo evaluate whether new/unique tooling and test equipment will be required to produce proposed alternatives and to understand overall requirements of tooling and test equipment as it will impact the cost and performance risk. To continue risk planning to address identified risks. Contractors need to establish tooling and test equipment concepts early in the program life-cycle and determine whether new/unique tooling and test equipment will be needed to produce the proposed alternative. If so, the risk must be addressed to achieving program requirements.\n\n\nSources of Information:  \nThis data should come from both design and manufacturing engineering working together along with test engineering, manufacturing tooling and test departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Prototype tooling and STE/SIE requirements been identified and documented\n•  Schedule for the tooling and STE/SIE requirements is documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have the requirements for tooling, Special Test Equipment, and/or Special Inspection Equipment been identified?\n2.  Do the identified requirements for tooling, Special Test Equipment, and/or Special Inspection Equipment include a supporting rationale?\n3.  Has a schedule for required tooling, Special Test Equipment, and/or Special Inspection Equipment been developed?\n\n\nAdditional Considerations:  \n•  None\n\n\nLessons Learned: \n•  Tooling and test equipment can be major cost and risk drivers and must be addressed in the AoA.  If new/unique tooling and test equipment is required, this must be tracked through EMD.\n"]
["$H$25",297,"Prototype tooling and STE/SIE concepts demonstrated in production relevant environment. Requirements development efforts for production tooling and STE/SIE complete.","Purpose:  \nTo understand what new/unique tooling and test equipment is required for a program by building prototype units in a production relevant environment. To determine the impact this equipment will have on program cost, schedule, and performance.  All of this information is needed to understand cost and performance risk for the selected alternative and to begin risk management effort in the subsequent phases.  Contractors need to establish tooling and test equipment concepts early in the program life-cycle and determine whether new/unique tooling and test equipment will be needed to produce the proposed alternative. If so, the risk must be addressed to achieving program requirements.\n\n\nSources of Information:  \nThis data should come from both design and manufacturing engineering working together along with test engineering, manufacturing tooling and test departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Prototype tooling and STE/SIE concepts demonstration documentation\n•  Production tooling and STE/SIE requirements documentation\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have prototype tooling, Special Test Equipment, and/or Special Inspection Equipment concepts been demonstrated in production relevant environment?  \n2.  Are efforts to develop requirements for production tooling, Special Test Equipment, and/or Special Inspection Equipment complete?  \n\n\nAdditional Considerations:  \n•  None\n\n\nLessons Learned: \n•  Tooling and test equipment can be major cost and risk drivers and should have been addressed in the AoA. If new/unique tooling and test equipment are required it must be demonstrated in a production relevant environment. \n•  It is very important that the customer work with the contractor to determine what defines a production relevant environment.\n"]
["$I$25",298,"Design and development efforts for production tooling and STE/SIE initiated with STE/SIE validation plans complete. Manufacturing equipment maintenance strategy developed.","Purpose:  \nTo demonstrate the production tooling and test equipment by building early EMD units in a production representative environment. To ensure that production tooling and test equipment design and development is progressing so it can be used on the pilot line. To begin planning for overall maintenance requirements for tooling and test equipment. This is the beginning of the effort to understand manufacturing capability and risk in EMD. Contractors must demonstrate manufacturing capability in this phase, and tooling and test equipment is a key element of this activity. The development of required tooling and test equipment must be monitored closely, and using EMD builds can be a good approach to monitoring progress.\n\n\nSources of Information:  \nThis data should come from both design and manufacturing engineering working together along with test engineering, manufacturing tooling and test departments.\n\n\nExamples of ‘Objective Evidence’:\n•Production tooling and STE/SIE design and development efforts underway\n•Manufacturing equipment maintenance strategy developed\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have design and development efforts for production tooling, Special Test Equipment, and/or Special Inspection Equipment been initiated?\n2. Are Special Test Equipment, and/or Special Inspection Equipment validation plans complete?\n3. Has a manufacturing equipment (including tooling, Special Test Equipment, and/or Special Inspection Equipment) maintenance strategy been developed? \n\n\n\nAdditional Considerations:  \n•Is there a documented record of calibration and certification of STE/SIE?\n\n\nLessons Learned: \n•The monitoring of the design, development and use of the tooling and test equipment needs to be accomplished because this equipment can be major cost and risk drivers. \n•Making sure the contractor has a maintenance strategy is important to ensure good product, and this area typically does not get addressed until there is a problem.\n•Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals.\n"]
["$J$25",299,"Tooling, test and inspection equipment proven on pilot line and additional requirements identified for LRIP.  STE/SIE validated as part of pilot line validation IAW validation plans.  Mfg equipment maintenance demonstrated on pilot line.","Purpose:  \nTo demonstrate that tooling and test equipment used on a pilot production line can meet program requirements. During this phase maintenance requirements on the tooling and test equipment must be demonstrated.  All tooling and test equipment required for LRIP must be identified, and to the maximum extent practicable, demonstrated on the pilot line.  If not, you must assess the risk of moving forward and ensure decision makers are aware of the risk and mitigation options. The program must demonstrate manufacturing capability prior to proceeding into production and tooling and test equipment are an essential element for that demonstration.\n\n\nSources of Information:  \nThis data should come from both design and manufacturing engineering working together along with test engineering, manufacturing tooling and test departments.\n\n\nExamples of ‘Objective Evidence’:\n•All tooling, test, and inspection equipment proven on the pilot line \n•All tooling, test and inspection equipment requirements identified for Low Rate Initial Production (LRIP)\n•Manufacturing equipment maintenance demonstrated on pilot line \n•All STE/SIE validated as part of pilot line validation IAW validation plan\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have all tooling, test, and inspection equipment been proven on the pilot line?\n2. Have additional tooling, test and inspection equipment requirements been identified for Low Rate Initial Production (LRIP)? \n3. Have Special Test Equipment, and/or Special Inspection Equipment been validated as part of pilot line and IAW validation in accordance with the validation plan? \n4. Have maintenance process and procedures for manufacturing equipment (including tooling, Special Test Equipment, and/or Special Inspection Equipment) been demonstrated on pilot line?\n\n\n\nAdditional Considerations:  \n•Is there a documented record of calibration and certification of STE/SIE?\n\n\nLessons Learned: \n•Tooling and test equipment can be major cost and risk drivers, and their capability needs to be fully understood before proceeding into Production. If there will be tooling and test equipment used in production that have not yet demonstrated the required performance, you must mitigate the risk.  \n•Examine the rate capability and utilization of EMD tooling and test equipment to ensure there is adequate capacity.  If insufficient, ensure that the program has identified and budgeted money to obtain the required tooling and test equipment for LRIP. \n•In the past, program managers have assumed there was adequate test equipment capacity available using EMD assets, which turned out not to be case because of assets needed to support Operation Test and Evaluation (OT&E) or other test efforts. \n•Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals.\n"]
["$K$25",300,"All tooling, test and inspection equipment proven in LRIP and additional requirements identified for FRP.  Manufacturing equipment maintenance schedule demonstrated. STE/SIE validation maintained as necessary.","Purpose:  \nTo demonstrate that the tooling and test equipment have been adequately proven to proceed into Full Rate Production and that the maintenance schedule will support production requirements. The program is required to demonstrate there is no significant manufacturing risk prior to proceeding to FRP. The capability and capacity of tooling and test equipment are an essential element in that demonstration.   \n\n\nSources of Information:  \nThis data should come from both design and manufacturing engineering working together along with test engineering, manufacturing tooling and test departments.\n \n\nExamples of ‘Objective Evidence’:\n•Evidence that all tooling, test, and inspection equipment were proven in LRIP\n•FRP tooling and STE/SIE requirements documentation\n•Manufacturing equipment maintenance plans\n•STE/SIE validation and revalidation documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has all tooling, test, and inspection equipment (i.e., STE/SIE) been proven in the Low Rate Initial Production (LRIP)?\n2. Have additional tooling, test, and inspection equipment (i.e., STE/SIE) requirements been identified for Full Rate Production (FRP)?\n3. Have maintenance process and procedures for manufacturing equipment (including tooling, Special Test Equipment, and/or Special Inspection Equipment) been demonstrated during Low Rate Initial Production (LRIP)?\n4. Has validation of Special Test Equipment, and/or Special Inspection Equipment (STE/SIE) been maintained during Low Rate Initial Production (LRIP)?\n\n\n\nAdditional Considerations:  \n• Is there a documented record of calibration and certification of STE/SIE?\n\n\nLessons Learned: \n• The capability and capacity of the tooling and test equipment can be major cost and risk drivers and must be addressed when you are demonstrating that there is no significant manufacturing risk for Full Rate Production.  \n• Look at key elements in this area (e.g., cycle time, yield, scrap and rework, etc.) when you assess the capability to achieve program requirements and determining capacity. You must look at all of these elements to determine rate requirements for tooling and test equipment.\n• Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals."]
["$L$25",301,"Proven tooling, test and inspection equipment in place to support maximum FRP.  Planned equipment maintenance schedule achieved. STE/SIE validation maintained as necessary.","Purpose:  \nTo proceed to Full Rate Production (i.e., you have demonstrated the capability and capacity to support program requirements and that the maintenance schedule will support production. The program is required to demonstrate control of the manufacturing process, and the tooling and test equipment is a critical part of that demonstration. \n\n\nSources of Information:  \nThis data should come from both design and manufacturing engineering working together along with test engineering, manufacturing tooling and test departments.\n \n\nExamples of ‘Objective Evidence’:\n•Tooling and STE/SIE demonstrated for FRP and documented\n•Planned equipment maintenance documented\n•Evidence that STE/SIE validation and revalidation has been completed\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Is all tooling, test, and inspection equipment (i.e., STE/SIE) for Full Rate Production (FRP) in place? \n2. Has a planned equipment maintenance schedule been achieved?\n3. Has Special Test Equipment (STE) / Special Inspection equipment (SIE) validation been maintained as necessary?\n\n\n\nAdditional Considerations:  \n• Is sufficient tooling and test equipment available, or can it be obtained, to meet surge requirements, if necessary?\n• Is there a documented record of calibration and certification of STE/SIE?\n\n\nLessons Learned: \n• The capability and capacity of the tooling and test equipment can be major cost and risk drivers and must be addressed when you are demonstrating that there is no significant manufacturing risk for Full Rate Production.  \n• Look at key elements in this area (e.g., cycle time, yield, scrap and rework, etc.) when you assess the capability to achieve program requirements and determining capacity. You must look at all of these elements to determine rate requirements for tooling and test equipment.\n• Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals."]
["$A$26",302]
["$B$26",303,"H.2 - Facilities"]
["$C$26",304,"Current facility capabilities and capacity surveyed.","Purpose:  \nTo identify whether current facility capabilities and capacity requirements for the manufacturing state of the art. To use this information to understand cost and performance risk of selecting the best alternative and to begin risk planning to address potential risks. The contractor must determine whether facility capabilities and capacity are available to produce items being considered. If additional facility capabilities and capacity will be needed, you must begin addressing the risk to achieving program requirements.\n\n\nSources of Information:  \nThis data should come from facilities, design and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Current facility capabilities and capacity surveyed and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have current facility capabilities and capacity been surveyed? \n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$26",305,"Potential facility capabilities and capacity requirements identified.","Purpose:  \nTo identify whether potential facility capabilities and capacity requirements for characterizing the state of the manufacturing state of the art. To use this information to understand cost and performance risk of selecting the best alternative and to begin risk planning to address potential risks. The contractor must determine whether facility capabilities and capacity are available to produce items being considered. If additional facility capabilities and capacity will be needed, you must begin addressing the risk to achieving program requirements.\n\nSources of Information:  \nThis data should come from facilities, design and manufacturing departments.\n\nExamples of ‘Objective Evidence’:\n•  Potential facility capabilities and capacity requirements for concepts identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have potential facility capabilities and capacity requirements been identified?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$E$26",306,"Facility capabilities and capacity requirements and gaps for system concepts identified. ","Purpose:  \nTo evaluate whether new/specialized facilities will be required to produce proposed alternatives. To use this information to understand cost and performance risk of selecting the best alternative and to begin risk planning to address identified risks. The contractor must determine whether specialized facilities are needed to produce items being considered. If specialized facilities will be needed, you must begin addressing the risk to achieving program requirements\n\nSources of Information:  \nThis data should come from facilities, design and manufacturing departments.\n \nExamples of ‘Objective Evidence’:\n•  Facility capabilities and capacity requirements and gaps for system concepts identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have facility capabilities and capacity requirements and gaps for system concepts been identified?\n\nAdditional Considerations:  \n• None \n\nLessons Learned: \n• None"]
["$F$26",307,"Capability and availability of manufacturing facilities for prototype development and production of the preferred materiel solution evaluated, included in the AoA, and documented in the AS and SEP. Human factors & ergonomics and safety requirements for manufacturing (personnel, processes & equipment) identified.","Purpose:  \nTo identify facilities that will be required to produce proposed alternatives and to identify the facility requirements needed to produce pilot line items in EMD. To use this information to understand cost and performance risk in selecting the best alternative and to begin planning to address identified risks. Contractors must determine what facilities are required to produce prototype hardware very early in the product life-cycle and also plan where they will produce the units later in production. The lead time to address facilities can be extremely long, and if you plan to use different facilities (e.g., one facility to fabricate prototypes and one for production) you must understand the risk early in the program and make the decision of which facility to select and modify. \n\n\nSources of Information:  \nThis data should come from engineering, facilities and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Availability of manufacturing facilities for prototype development and production evaluated and documented \n•  Analysis of Alternatives (AoA) includes availability of manufacturing facilities for prototype development and production\n•  AS and SEP includes availability of manufacturing facilities for prototype development and production\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have capability and availability of manufacturing facilities for prototype development and production of the preferred materiel solution been evaluated?\n2. Have capability and availability of manufacturing facilities for prototype development and production of the preferred materiel solution been included in the AoA?\n3. Have capability and availability of manufacturing facilities for prototype development and production of the preferred materiel solution been documented in the AS and SEP?\n4. Has human factors & ergonomics and safety requirements for manufacturing (personnel, processes & equipment) been identified?\n\n\nAdditional Considerations:  \n• Is available capital sufficient for required facilities, test equipment and tooling?\n\n\nLessons Learned: \n• Due to cost and schedule risk in this area, focus on both near term facility needs to produce prototypes and long term needs for production units. The facilities used to produce the prototype units are sometimes different from those used for production, or required production facilities may be significantly larger. If either situation occurs, it requires risk assessment and management to minimize the impact. The biggest issues are cost and schedule. The contractor must ensure he has sufficient capital to acquire the new facilities and that he starts his effort in adequate lead time to ensure new facilities are available when needed. \n• When you move to a new facility you need to address lost learning and ensure engineering and manufacturing support is in place.\n"]
["$G$26",308,"Manufacturing facilities identified and plans developed to produce prototypes. Human factors & ergonomics and safety requirements for manufacturing (personnel, processes & equipment) assessed.","Purpose:  \nTo evaluate the availability of the manufacturing facilities that will be required to produce the alternatives being considered. To use this information to understand the performance risk in selecting the best alternative and to begin risk planning to address those risk identified. There should be a discussion on how planned facilities to produce the prototype units will meet the requirement to build items in a production relevant environment.  If required facilities are not readily available to produce the prototype units, this area must be closely monitored to ensure the TMRR schedule can be met.  \n\n\nSources of Information:  \nThis data should come from engineering, facilities and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Documentation for the identification of manufacturing facilities to produce prototypes \n•  Plans for the manufacturing facilities to produce prototypes\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have manufacturing facilities that meet requirements to produce prototypes been identified?\n2.  Have plans to produce prototypes in identified facilities been developed?\n3.  Have requirements for human factors and ergonomics for manufacturing personnel, processes, and equipment in identified facilities been assessed?\n\n\nAdditional Considerations:  \n•  Have sufficient funds been budgeted for required facilities, test equipment and tooling?\n•  It is very important to begin discussions in this phase to identify what is a production relevant environment for facilities.\n\n\nLessons Learned: \n•  The facilities used for the prototype must be assessed from both the perspective of producing prototypes and how they will simulate a production relevant environment. \n•  In the past is that we did not adequately understand the risk of transitioning from one facility (e.g., laboratory) to a production environment. There is no requirement that facilities being used in this phase be the same as production, but it is important to understand the risk of transitioning to a new facility.\n"]
["$H$26",309,"Manufacturing facilities identified and plans developed to produce pilot line build. Human factors & ergonomics and safety requirements for manufacturing (personnel, processes & equipment) verified in a production relevant environment.","Purpose:  \nTo understand the planning needed for the right facilities to demonstrate pilot line production.  This information is needed to determine risk of proceeding into EMD. Due to the typical lead time and cost associated with facilities, it is important to address these issues as soon as possible.\n\n\nSources of Information:  \nThis data should come from design, facilities, and manufacturing offices.\n\n\nExamples of ‘Objective Evidence’:\n•  Manufacturing plan\n•  Plan for producing the pilot line build\n•  Shop floor ESH plans\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have manufacturing facilities that meet requirements for pilot line been identified?\n2.  Have plans for building pilot line in identified facilities been developed?\n3.  Have requirements for human factors and ergonomics for manufacturing personnel, processes, and equipment been verified in a production relevant environment?\n\n\nAdditional Considerations: \n•  Has the budget been approved for required facilities, test equipment and tooling?\n\n\nLessons Learned: \n•  In the past we have not adequately assessed facility requirements for demonstrating production early enough in the acquisition process.  This has resulted in extending the EMD phase and/or adding significant risk as we transition into production.\n"]
["$I$26",310,"Manufacturing facilities identified and plans developed to produce LRIP build. Human factors & ergonomics and safety practices for manufacturing (personnel, processes & equipment) validated in a production representative environment. ","Purpose:  \nTo demonstrate facilities are adequate by building the early EMD units in a production representative environment. To ensure the Low Rate Initial Production (LRIP) facility planning effort is progressing satisfactorily to support LRIP requirements. Contractors must demonstrate manufacturing capability in this phase, and facilities are part of that demonstration. Acquiring facilities may require significant time and money; therefore, this effort needs to be monitored closely focusing on production of EMD units and planning for LRIP.\n\n\nSources of Information:  \nThis data should come from design, facilities, and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Manufacturing plan\n•Manufacturing facilities identified to produce the Low Rate Initial Production (LRIP) build \n•Plans for the identified manufacturing facilities developed to produce the Low Rate Initial Production (LRIP) build\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have manufacturing facilities been identified to produce the Low Rate Initial Production (LRIP) build?\n2. Have plans for the identified manufacturing facilities been developed to produce the Low Rate Initial Production (LRIP) build?\n3. Have requirements for human factors and ergonomics for manufacturing personnel, processes, and equipment been validated in a production representative environment?\n4. Have safety requirements for manufacturing personnel, processes, and equipment been validated in a production representative environment?\n\n\nAdditional Considerations: \n•Have funds been programmed or expended for necessary facilities, equipment and tooling?\n\n\nLessons Learned: \n•Facilities can be a major cost and schedule driver, so they need to be addressed in assessment of manufacturing capability. The monitoring of the facilities planning and progress needed to support the LRIP requirements must be a focus area, especially if new facilities are required or plans are being made to move to a different facility in LRIP. \n•You should start using planned production facilities as soon as practical in EMD.\n"]
["$J$26",311,"Pilot line facilities demonstrated. Manufacturing facilities adequate to begin LRIP.  Plans in place to support transition to FRP. Workplace safety is adequate. Human factors & ergonomics and safety practices for manufacturing (personnel, processes & equipment) demonstrated on a pilot line.","Purpose:  \nTo demonstrate that the pilot line can achieve program LRIP requirements.  To ensure all the facilities required for LRIP are identified and, to the maximum extent practicable, demonstrated in the pilot line.  If all facilities planned for LRIP cannot be demonstrated, you must assess the risk of moving forward and ensure decision-makers are aware of the risk and mitigation options. Finally, you need to assess planning to have adequate facilities for Full Rate Production (FRP). The program is required to demonstrate manufacturing capability prior to Milestone C. The facilities needed for LRIP must be considered in making that determination.   Due to the typical lead times and cost associated with facilities, it is important to address the requirements for FRP before you exit EMD.\n\n\nSources of Information:  \nThis data should come from design, facilities, and manufacturing offices.\n\n\nExamples of ‘Objective Evidence’:\n•Manufacturing plan\n•Pilot line facilities demonstrated \n•Plan in place to support the transition to Full Rate Production (FRP)\n•Workplace safety adequate – Safety records\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have pilot line facilities been successfully demonstrated?\n2. Are manufacturing facilities adequate and ready to begin Low Rate Initial Production (LRIP)? \n3. Are plans in place to support manufacturing facilities transition to Full Rate Production (FRP)?\n4. Is workplace safety meeting requirements (i.e., adequate)?\n5. Have human factors and ergonomics for manufacturing personnel, processes, and equipment been demonstrated on a pilot line?\n6. Have safety requirements for manufacturing personnel, processes, and equipment been demonstrated on a pilot line?\n\n\nAdditional Considerations: \n•New facility construction, if required, should be virtually complete at this time.\n\n\nLessons Learned: \n•New facilities can be major cost and risk drivers, and their capability must be fully understood before proceeding into Production. If there are facilities required in production that have not yet been demonstrated in EMD, you must ensure risk management is applied. \n•You also need to verify rate capability and utilization of EMD facilities to be sure there is adequate capacity in those facilities or that the program manager has identified and has capital to obtain additional facilities for LRIP.\n"]
["$K$26",312,"Manufacturing facilities in place and demonstrated in LRIP.   Capacity plans adequate to support FRP. Human factors & ergonomics and safety practices for manufacturing (personnel, processes & equipment) demonstrated in LRIP .","Purpose:  \nTo demonstrate there is no significant manufacturing risk prior to proceeding into FRP. The capability and capacity of your facilities are an essential element in making that determination.   \n\n\nSources of Information:  \nThis data should come from both design, facilities, and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Manufacturing plan\n•Manufacturing facilities demonstrated for LRIP \n•Capacity Plan(s) to support FRP\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have manufacturing facilities been successfully demonstrated in Low Rate Initial Production (LRIP)?\n2. Are manufacturing facility capacity plans adequate to meet the requirements for Full Rate Production (FRP)? \n3. Have human factors and ergonomics for manufacturing personnel, processes, and equipment been demonstrated during LRIP?\n4. Have safety requirements for manufacturing personnel, processes, and equipment been demonstrated during LRIP?\n\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• The capability and capacity of the facilities can be major cost and risk drivers and must be assessed if you wish to demonstrate that there is no significant manufacturing risk for Full Rate Production.  You must examine the major elements that drive capacity (e.g., cycle time, yield, scrap and rework, etc.) when you assess capacity to achieve program requirements."]
["$L$26",313,"Production facilities in place and capacity demonstrated to meet maximum FRP requirements. Human factors & ergonomics and safety requirements for manufacturing (personnel, processes & equipment) demonstrated in FRP.","Purpose:  \nTo demonstrate control of the manufacturing process and the facilities used for FRP. The capability and capacity of your facilities, as it relates to your program requirements, are an essential element in making that determination.   \n\n\nSources of Information:  \nThis data should come from design, facilities, and manufacturing departments.\n \n\nExamples of ‘Objective Evidence’:\n•Manufacturing plan\n•Facilities management plan in place to meet FRP build\n•Evidence that capacity of the production facilities have been demonstrated to meet FRP requirements\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are the production facilities in place to meet Full Rate Production (FRP) requirements?\n2. Has the capacity of the production facilities been demonstrated to meet maximum Full Rate Production (FRP) requirements?\n3. Have human factors and ergonomics for manufacturing personnel, processes, and equipment been demonstrated during FRP?\n4. Have safety requirements for manufacturing personnel, processes, and equipment been demonstrated during FRP?\n\n \n\nAdditional Considerations:  \n• Is sufficient facility capacity available, or can it be obtained, to meet surge requirements, if necessary?\n\n\nLessons Learned: \n• The capability and capacity of the facilities can be major cost and risk drivers and need to be addressed when you are demonstrating that that manufacturing process are under control for Full Rate Production.  You need to look at key elements in this area (e.g., cycle time, yield, scrap and rework, etc.) when you assess the capability and capacity. "]
["$A$27",314,"I - Mfg Management","Manufacturing Management: Requires an analysis of the orchestration of all elements needed to translate the design into an integrated and fielded system (meeting Program goals for affordability and availability).\n"]
["$B$27",315,"I.1 - Mfg Planning & Scheduling"]
["$C$27",316,"Manufacturing management considerations surveyed and included in early planning activities.","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity.  DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in the product development life cycle is a root cause of cost and schedule growth. Manufacturing readiness must be addressed early in the acquisition process and all phases of Science and Technology (S&T).  \n\n\nSources of Information:\nThis data should come from engineering and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Manufacturing management considerations surveyed and included in early planning activities and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have manufacturing management considerations been surveyed and included in early planning activities?\n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$27",317,"Manufacturing management needs assessed, analyzed and validated.","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity.  DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in the product development life cycle is a root cause of cost and schedule growth. There is near universal agreement that examining manufacturing maturity for the first time just prior to production is a mistake; manufacturing readiness must be addressed early in the acquisition process and all phases of Science and Technology (S&T).  \n\nSources of Information:\nThis data should come from engineering and manufacturing departments.\n\nExamples of ‘Objective Evidence’:\n•  Manufacturing management needs for system concepts assessed, analyzed and validated and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have manufacturing management needs been assessed, analyzed and validated?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$E$27",318,"Manufacturing management requirements for system concepts identified. ","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity.  DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in the product life cycle is a root cause of cost and schedule growth. There is near universal agreement that examining manufacturing maturity for the first time just prior to production is a mistake; manufacturing readiness must be addressed early in the acquisition process and all phases of Science and Technology (S&T).  \n\nSources of Information:\nThis data should come from engineering and manufacturing departments.\n\nExamples of ‘Objective Evidence’:\n•  Manufacturing management requirements for system concepts identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have manufacturing management requirements for system concepts been identified?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$F$27",319,"Manufacturing strategy for the preferred materiel solution developed, considered in the AoA, and documented in the AS. Prototype schedule risk mitigation efforts documented in the SEP. ","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity.  DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in a program’s life cycle is a root cause of cost and schedule growth. There is near universal agreement that examining manufacturing maturity for the first time just prior to production is a mistake; manufacturing readiness must be addressed early in the acquisition process and some phases of Science and Technology (S&T).  \n\n\nSources of Information:\nThis data should come from engineering and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Manufacturing strategy for the preferred materiel solution documented \n•  AoA includes the manufacturing strategy for the preferred materiel solution\n•  AS includes the manufacturing strategy for the preferred materiel solution\n•  Prototype schedule risk mitigation efforts documented in the SEP\n\nQuestions (MRL Criteria in the form of a question):\n1. Has the manufacturing strategy for the preferred materiel solution been developed?\n2. Has the manufacturing strategy for the preferred materiel solution been considered in the AoA?\n3. Has the manufacturing strategy for the preferred materiel solution been incorporated into the Acquisition Strategy?\n4. Have the prototype schedule risk mitigation efforts been documented in the SEP?\n\nAdditional Considerations:\n•  Has a make-or-buy plan been developed for critical items at this stage?\n\n\nLessons Learned:\n•  During the MSA Phase, an assessment of manufacturing feasibility should be conducted for each competing materiel solution being examined in the AoA.  \n•  The manufacturing risk of each proposed materiel solution should be assessed to determine whether it can achieve program cost, schedule, and performance objectives.  The program should then begin addressing and mitigating any identified risk.\n"]
["$G$27",320,"Mfg. strategy refined based upon preferred concept.  Prototype schedule risk mitigation efforts initiated.  ","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity.  To identify and assess the manufacturing risk of the alternatives being considered and to begin maturing manufacturing processes to reduce risk.  One of the major activities in maturing processes in TMRR is to assess the degree to which the system being considered has been adequately demonstrated in a production relevant environment.  DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in a program’s life cycle is a root cause of cost and schedule growth. Manufacturing readiness must be addressed early in the acquisition process and some phases of Science and Technology (S&T).  \n\n\nSources of Information:\nThis data should come from engineering and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Documentation for manufacturing strategy refined based upon the preferred concept \n•  Documented prototype schedule risk mitigation efforts initiated \n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Based upon the preferred concept, has the manufacturing strategy been refined?\n2.  Have mitigation efforts concerning schedule risk of prototypes (i.e., testing and delivery schedules) been initiated?\n\n\nAdditional Considerations:\n•  A make-or-buy plan should have been developed for critical items at this stage.\n\n\nLessons Learned:\n•  The MRL process is a best practice to capture and manage manufacturing risk before proceeding to EMD.  While it is unlikely that contractors would have a complete factory and supply chain established this early in a program, there is still a need to obtain knowledge of critical manufacturing processes, production scale-up efforts, potential supply chain issues, and manufacturing maturity issues. \n•  It is essential that a good baseline of the manufacturing maturity and risk be established at the beginning of TMRR in order to employ correct risk mitigation efforts. If your program went through MSA, the results would be a good start in addressing the manufacturing risk; nevertheless, it is recommended that a manufacturing assessment be conducted early TMRR to establish a baseline for manufacturing maturity and to determine manufacturing risk. \n•  A major goal in TMRR is to mature the manufacturing process, but even more important, to identify, assess, and mitigate manufacturing risk before proceeding into EMD.\n"]
["$H$27",321,"Initial manufacturing approach developed.  All system design related manufacturing events included in IMP/IMS.  Manufacturing risk, and issue mitigation approach for pilot line and/or technology insertion programs defined.","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity.  To identify and assess the manufacturing risk of the alternatives being considered and to begin maturing the manufacturing processes to reduce risk.  One of the major activities in maturing processes in TMRR is to assess the degree to which the system being considered has been adequately demonstrated in a production relevant environment. \n\n\nSources of Information:\nThis data should come from engineering and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Initial manufacturing approach documentation\n•  Integrated Master Plan/Schedule (IMP/S)\n•  Documented program risk process\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Has the initial manufacturing approach for the next phase (EMD) been developed?\n2.  Are all system design related manufacturing events included in the Integrated Master Plan/Schedule (IMP/IMS)?\n3.  For either the pilot line and/or technology insertion, has a mitigation approach for manufacturing risks and issues been defined?\n\n\nAdditional Considerations:\n•  A make-or-buy plan should be developed for all BOM items at this stage.\n\n\nLessons Learned:\n•  The MRL process is a best practice to capture and manage manufacturing risk before proceeding to EMD.  While it is unlikely that contractors would have a complete factory and supply chain established this early in a program, there is still a need to obtain knowledge of critical manufacturing processes, production scale-up efforts, potential supply chain issues, and manufacturing maturity issues. \n•  It is essential that a good baseline of the manufacturing maturity and risk be established at the beginning of TMRR in order to employ the correct risk mitigation efforts. If your program had an MSA phase, the results from that would be a good start in addressing the manufacturing risk, but it is recommended in any case that you conduct a manufacturing assessment early in TMRR to establish a baseline for manufacturing maturity and determine the manufacturing risk. \n•  A major goal in this phase is to mature the manufacturing process, but even more important, to identify, assess, and mitigate manufacturing risk before proceeding to EMD.\n"]
["$I$27",322,"Initial Manufacturing Plan developed and included in IMP/IMS. Manufacturing risks and issues integrated into mitigation plans.   Initial work instructions developed.  Effective production control system in place to support pilot line. ","Purpose: \nTo build qualification test hardware on a Pilot Line to validate the manufacturing processes associated with the design.  The initial manufacturing planning to accomplish this should be evident.  Proper manufacturing planning and scheduling is necessary to accomplish these requirements.   The need for a Manufacturing Plan is based on the Best Practices approach identified in the Willoughby Templates (DOD 4245.7-M) and the NAVSO P-6071.  These documents explain that the Manufacturing Plan shall be sufficiently comprehensive so there is a high degree of confidence that the contractor has adequately evaluated and planned for production. The Manufacturing Plan should be used to monitor the contractual effort to ensure the timely and effective execution of the production program.  These documents also stress the importance of managing the manufacturing approach of major subcontractors/suppliers to the success of the overall system. \n\n\nSources of Information:\nThis data should come from engineering and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Initial manufacturing plan developed\n•Manufacturing planning required to achieve MRL 8 included in Integrated Master Plan/Schedule (IMP/S)\n•Manufacturing risks integrated into risk mitigation plans \n•Initial work instructions developed \n•Production control system in place to support the pilot line \n•Documented program risk process\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has the initial manufacturing plan been developed?\n2. Have manufacturing events from the initial Manufacturing Plan been included in the Integrated Master Plan and/or the Integrated Master Schedule?\n3. Have manufacturing risks and issues been integrated into program mitigation plans?\n4. Have initial work instructions been developed? \n5. Is an effective production control system in place to support the pilot line?\n\n\n\nAdditional Considerations:\n•Manufacturing Plan – The Manufacturing Plan documents the approach for fabricating a product in a cost-effective manner.  It is based on the results of detailed planning and analyses conducted to define the optimum approach for product manufacture.  All actions required to produce, test, and deliver acceptable systems on schedule at minimum cost should be documented in the plan.  The Manufacturing plan should address aspects such as process flows, scheduling, tooling, test equipment, plant facilities, material decisions, manpower, manufacturing risk management, producibility, etc.  A draft Manufacturing Plan should be delivered to the Government at the CDR.\n•Scheduling – The manufacturing schedule should provide assurance that the necessary resources will be available when needed, that no resources will be overloaded or expended during execution of any manufacturing tasks, and that product delivery dates are indeed achievable.  The planning needed to complete the Pilot Line should be included in the Integrated Master Schedule (IMS).\n•Manufacturing Flow – Preliminary assembly and test flows should be developed to show how the product will move through the manufacturing plant from the stock room to the shipping deck.  Activities such as Value Stream Mapping are extremely valuable in developing an efficient flow.  This activity is useful in determining the need for resources and in determining if the manufacturing schedule is achievable.\n•Manufacturing Documentation – Adequate manufacturing documentation is necessary to ensure processes, assembly, and test operations are performed correctly.  Manufacturing documentation includes items such as work instructions, test procedures, inspection procedures, kitting procedures, routers, travelers, tooling & test equipment lists, etc.  Preliminary documentation should be completed such that Pilot line build can be effectively accomplished.    \n•Manufacturing Risks – Manufacturing risks should have been identified and fed into the overall system risk management process.  Proposed solutions should be verified in the Pilot Line build and system qualification tests.\n\n\nLessons Learned:\n•While a formal Manufacturing Plan is usually required as a contract deliverable item at the end of EMD, be careful not to wait until the end of EMD to address this area.  To be effective, the manufacturing planning must be initiated at the beginning of EMD and proceed concurrently with the product design activities.  The contract deliverable requirements should require a draft Manufacturing Plan to be delivered at the CDR.  \n•Due to the fact that many prime contractors receive major components and assemblies from subcontractors, it is imperative that the requirements for proper manufacturing planning and scheduling be flowed down to subcontractors.\n"]
["$J$27",323,"Manufacturing Plan updated for LRIP.  All manufacturing risks and issues identified and assessed with approved mitigation plans in place. Work instructions finalized. Effective production control system in place to support LRIP.","Purpose: \nTo ensure that an adequate manufacturing approach has been developed for duplicating a product configuration in a cost-effective manner, and that any risks to achieving this goal have been identified and addressed. DoDI 5000.02 requires the development of an affordable and executable process, maturity of critical manufacturing processes, and the identification of manufacturing risks.  The need for a Manufacturing Plan is based on the Best Practices approach identified in the Willoughby Templates (DOD 4245.7-M) and the NAVSO P-6071.  These documents explain that the Manufacturing Plan shall be sufficiently comprehensive so there is a high degree of confidence that the contractor has adequately evaluated and planned for production. The Manufacturing Plan should be used to monitor the contractual effort to ensure the timely and effective execution of the production program.  These documents also stress the importance of managing the manufacturing approach of major subcontractors/suppliers to the success of the overall system. \n\n\nSources of Information:\nThe majority of the information should be available in the Manufacturing Plan and production floor manufacturing documentation.  Also review the system risk management plans and results.\n\n\nExamples of ‘Objective Evidence’:\n•Manufacturing plan updated for Low Rate Initial Production (LRIP)\n•All key manufacturing risks identified and assessed with approved mitigation plans in place \n•Work instructions in place and finalized \n•An effective production control system is in place to support Low Rate Initial Production (LRIP)\n•Integrated Master Plan/Schedule (IMP/S)\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has the manufacturing plan been updated for Low Rate Initial Production (LRIP)?\n2. Have all manufacturing risks and issues been identified and assessed?\n3. Do all identified manufacturing risks and issues have approved mitigation plans in place?\n3. Have work instructions been finalized? \n4. Is an effective production control system in place to support Low Rate Initial Production (LRIP)?\n\n\n\nAdditional Considerations:\n•Is the overall manufacturing approach adequate for LRIP?\n•Have all manufacturing risks been identified with mitigation plan developed?\n•Are manufacturing risks incorporated into the overall risk management program?\n•Has a manufacturing flow been developed?\n•Has a Manufacturing Plan been developed to reflect the LRIP build?\n•Have all Government furnished facilities, property, materials, and equipment been identified?\n•Have all work instructions, kitting procedures, shop floor travelers, etc. been developed?\n•Is this documentation under formal configuration control?\n•Has this documentation been demonstrated on Pilot Line build?\n•Are work instructions available and/or posted on the factory floor at the point of use?\n•How frequently are work instructions reviewed and revised?\n•Are tooling designators clearly marked on the tooling to be used and referenced in the work instructions?\n•Do work instructions clearly and fully describe the assembly procedure?\n•Do work instructions contain visual aids?\n•Are all of the work in progress (WIP) and material used in assembly properly labeled so the status of each part and material can be readily ascertained?  (Status would include which operations have been performed on the WIP and expiration data for shelf-life limited material.)\n•Is WIP properly protected against damage during the manufacturing process?  \n•Is production material properly maintained against adverse effects of extreme heat or cold, moisture, ultraviolet light, unacceptable particulate matter (dust, dirt, grime, as applicable), or any other condition which would compromise the material?\n\n\nLessons Learned:\n•While a formal Manufacturing Plan is usually required as a contract deliverable item at the end of EMD, be careful not to wait until the end of EMD to address this area.  To be effective, the manufacturing planning must be initiated at the beginning of EMD and proceed concurrently with the product design activities.  The contract deliverable requirements should require a draft Manufacturing Plan to be delivered at the CDR.  \n• Due to the fact that many prime contractors receive major components and assemblies from subcontractors, it is imperative that the requirements for proper manufacturing planning and scheduling be flowed down to subcontractors.  \n"]
["$K$27",324,"Manufacturing plan updated for FRP.  All manufacturing risks and issues managed. Effective production control system in place to support FRP.","Purpose: \nTo ensure that an adequate manufacturing approach has been demonstrated in a low rate production environment and that adequate planning is available for cost-effective production at high rates.  DoDI 5000.02 requires “demonstrated control of the manufacturing process”.\n\n\nSources of Information:\nThe majority of the information should be available in the Manufacturing Plan and production floor manufacturing documentation.  Also review the system risk management plans and results. \n\n\nExamples of ‘Objective Evidence’:\n•System risk management plans and mitigation \n•Manufacturing plan updated for FRP\n•Evidence that an effective production control system is in place to support FRP\n•Documented program risk process\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has the manufacturing plan been updated for Full Rate Production (FRP)?\n2. Are all manufacturing risks and issues being managed?\n3. Is an effective production control system in place to support Full Rate Production (FRP)?\n\n\n\nAdditional Considerations:\n• Is the overall manufacturing approach adequate for FRP?\n• Have all manufacturing risks been identified with mitigation plan developed?\n• Are manufacturing risks incorporated into the overall risk management program?\n• Has a manufacturing flow been developed?\n• Has a Manufacturing Plan been developed to reflect the FRP build?\n• Have all Government furnished facilities, property, materials, and equipment been identified?\n• Have all work instructions, kitting procedures, shop floor travelers, etc. been developed?\n• Is this documentation under formal configuration control?\n• Has this documentation been demonstrated on the LRIP build?\n• Are work instructions available and/or posted on the factory floor at the point of use?\n• How frequently are work instructions reviewed and revised?\n• Are tooling designators clearly marked on the tooling to be used and referenced in the work instructions?\n• Do work instructions clearly and fully describe the assembly procedure?\n• Do work instructions contain visual aids?\n• Are all of the work in progress (WIP) and material used in assembly properly labeled so the status of each part and material can be readily ascertained?  (Status would include which operations have been performed on the WIP and expiration data for shelf-life limited material.)\n•Is WIP properly protected against damage during the manufacturing process?  \n•Is production material properly maintained against adverse effects of extreme heat or cold, moisture, ultraviolet light, unacceptable particulate matter (dust, dirt, grime, as applicable), or any other condition which would compromise the material?\n\n\nLessons Learned:\n• Increasing the production rate capability to FRP requirements effects many aspects of the manufacturing operations.  Proactive planning is critical to ensure that efficient and economical production at these higher rates can be achieved.  \n• It is also critical to fully understand the characteristics of the manufacturing processes so that proper controls can be implemented, and any risks associated with increasing the production rate can be identified and mitigated.  \n• This proactive planning must also be flowed down to suppliers."]
["$L$27",325,"All manufacturing risks and issues managed.","Purpose: \nTo ensure that all manufacturing planning and scheduling activities have been demonstrated in a high rate production environment, and are sufficient for continued cost-effective production at high rates.  \n\n\nSources of Information:\nThis data should come from program, engineering and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Integrated Master Plan/Schedule (IMP/S)\n•Documented program risk process includes manufacturing risks\n•All manufacturing risks have approved MMPs\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Are all manufacturing risks and issues being managed?\n\n\nAdditional Considerations:\n• Has manufacturing planning been made for surge and/or other contingency production?\n\n\nLessons Learned:\n• Increasing the production rate capability to FRP requirements effects many aspects of the manufacturing operations.  Proactive planning is critical to ensure that efficient and economical production at these higher rates can be achieved.  \n• It is also critical to fully understand the characteristics of the manufacturing processes so that proper controls can be implemented, and any risks associated with increasing the production rate can be identified and mitigated.  \n• This proactive planning must also be flowed down to suppliers.  "]
["$A$28",326]
["$B$28",327,"I.2 - Materials Planning  "]
["$C$28",328,"Materials planning state of the art surveyed.","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity.  DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in the product life cycle is a root cause of cost and schedule growth. Manufacturing readiness must be addressed early in the acquisition process and all phases of Science and Technology (S&T).  \n\n\nSources of Information:\nThis data should come from engineering and manufacturing departments. \n\n\nExamples of ‘Objective Evidence’:\n•  Materials planning state of the art surveyed and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Has materials planning state of the art been surveyed? \n\n\nAdditional Considerations:  \n• None\n\n\nLessons Learned: \n• None\n"]
["$D$28",329,"Initial availability, lead time, handling and storage requirements for potential materials and components evaluated.","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity.  DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in the product life cycle is a root cause of cost and schedule growth. There is near universal agreement that examining manufacturing maturity for the first time just prior to production is a mistake; manufacturing readiness must be addressed early in the acquisition process and all phases of Science and Technology (S&T).  \n\nSources of Information:\nThis data should come from engineering and manufacturing departments. \n\nExamples of ‘Objective Evidence’:\n•  Initial availability, lead time, handling and storage requirements for potential materials and components have been evaluated and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Have initial availability, lead time, handling and storage requirements for potential materials and components been evaluated? \n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$E$28",330,"Materials and components list for system concepts developed. Initial materials planning requirements (i.e., availability, lead times, handling, and storage) identified.","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity.  DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in the product life cycle is a root cause of cost and schedule growth. There is near universal agreement that examining manufacturing maturity for the first time just prior to production is a mistake; manufacturing readiness must be addressed early in the acquisition process and all phases of Science and Technology (S&T).  \n\nSources of Information:\nThis data should come from engineering and manufacturing departments. \n\nExamples of ‘Objective Evidence’:\n•  A documented materials and components list for system concepts\n•  Initial materials planning requirements (i.e., availability, lead times, handling, and storage) identified and documented\n\nQuestions (MRL Criteria in the form of a question):\n1. Has a materials and components list for system concepts been developed?\n2. Have initial materials planning requirements (i.e., availability, lead times, handling, and storage) been identified?\n\nAdditional Considerations:  \n• None\n\nLessons Learned: \n• None\n"]
["$F$28",331,"Materials and components list with estimates for availability, lead times, handling and storage requirements developed and considered in the AoA. ","Purpose: \nTo ensure risks associated with material selection are being addressed.  DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in a program’s life cycle is a root cause of cost and schedule growth. There is near universal agreement that examining manufacturing maturity for the first time just prior to production is a mistake; manufacturing readiness must be addressed early in the acquisition process and some phases of Science and Technology (S&T).  Recognition of the need to address manufacturing readiness earlier in acquisition policy was implemented by Department of Defense Instruction 5000.02, Operation of the Defense Acquisition System. The policy established maturity criteria for measuring risks associated with manufacturing processes for Milestone A.\n\n\nSources of Information:\nThis data should come from engineering and manufacturing departments. \n\n\nExamples of ‘Objective Evidence’:\n•  AoA includes the materials and components list with estimates for availability, lead times, handling and storage requirements\n\nQuestions (MRL Criteria in the form of a question):\n1. Has the materials and components list with estimates for availability, lead times, handling and storage requirements been developed and considered in the AoA?\n\n\nAdditional Considerations:\n• None\n\n\nLessons Learned:\n•  During the MSA Phase, an assessment of manufacturing feasibility should be conducted for each competing materiel solution being examined in the AoA.  \n•  The manufacturing risk of each proposed materiel solution should be assessed to determine whether it can achieve program cost, schedule, and performance objectives.  The program should then begin addressing and mitigating any identified risk.  \n•  The list of material needed for Technology Development components should be developed, with associated lead time estimates."]
["$G$28",332,"Make/buy evaluations initiated and include production considerations for pilot line, LRIP, and FRP needs. Lead times and other materials risks and issues identified. ","Purpose: \nTo ensure risks associated with material selection are being addressed.  One of the key activities in maturing the manufacturing process in the TMRR phase is to assess the degree to which the materials being considered have been adequately demonstrated in a relevant environment. DOD case studies and GAO reports have demonstrated that failure to adequately manage manufacturing risk early in a program’s life cycle is a root cause of cost and schedule growth. There is near universal agreement that examining manufacturing maturity for the first time just prior to production is a mistake; manufacturing readiness must be addressed early in the acquisition process and some phases of Science and Technology (S&T).  Recognition of the need to address manufacturing readiness earlier in acquisition policy was implemented by Department of Defense Instruction 5000.02, Operation of the Defense Acquisition System. The policy established maturity criteria for measuring risks associated with manufacturing processes for Milestones B.\n\n\nSources of Information:\nThis data should come from engineering and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Documented technology development part list \n•  Documented make/buy evaluations \n•  Make/buy evaluations include production considerations reflecting the pilot line, Low Rate Initial Production (LRIP), and Full Rate Production (FRP) needs \n•  Lead times and other risks identified and documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have make or buy evaluations been initiated?\n2.  Do the make or buy evaluations include requirements and considerations for pilot line, initial production (LRIP), and rate production (FRP)? \n3.  Have lead times and other materials risks and issues been identified?\n\n\nAdditional Considerations:\n•  Have you checked GIDEP alerts for obsolete or DMSMS materials?\n\n\nLessons Learned:\n•  The material selection process should be ongoing.  The parts list for TMRR units should be maturing.  The Make/Buy evaluations should have begun.  These evaluations should consider which manufacturing capabilities are available at the prime contractor and which ones will need to be farmed out to suppliers.  \n•  The Make/Buy selections should also reflect the proposed manufacturing approach for the pilot line, LRIP, and FRP.   Lead times and other risks associated with material selection should be identified.\n"]
["$H$28",333,"Most material make/buy decisions complete, material risks and issues identified, and mitigation plans developed. BOM initiated. ","Purpose: \nTo ensure risks associated with material selection are being addressed.  One of the key activities in maturing the manufacturing process in the TMRR phase is to assess the degree to which the materials being considered have been adequately demonstrated in a manufacturing relevant environment. DoDI 5000.02 requires “The project shall exit the Technology Development Phase when an affordable program or increment of militarily useful capability has been identified; the technology and manufacturing processes for that program or increment have been assessed and demonstrated in a relevant environment; manufacturing risks have been identified; a system or increment can be developed for production within a short timeframe (normally less than 5 years for weapon systems).”   \n\n\nSources of Information:\nThis data should come from engineering and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•  Risk mitigation plans \n•  Manufacturing Maturation Plans (MMPs)\n•  BOM documentation\n•  Make/Buy decisions documented\n\n\nQuestions (MRL Criteria in the form of a question):\n1.  Have most make or buy trade studies been completed?\n2.  Have materials risks and issues been assessed (identified)? \n3.  Have mitigation plans for the identified materials risks and issues been developed?\n4.  Has the Bill of Materials (BOM) been initiated?\n\n\nAdditional Considerations:\n•  Have you checked GIDEP alerts for obsolete or DMSMS materials?\n\n\nLessons Learned:\n•  The material selection process should be well underway with significant maturity.  Most Make/Buy evaluations should be complete.  These evaluations should consider which manufacturing capabilities are available at the prime contractor and which ones will need to be farmed out to suppliers.  \n•  The Make/Buy selections should also reflect the proposed manufacturing approach for the pilot line, LRIP, and FRP.  Lead times and other risks associated with material selection should be identified.  The development of the BOM should be initiated.  \n•  A major goal in this phase is to mature the material selection process, but even more important, to identify, assess, and mitigate manufacturing risk before proceeding to EMD.\n•  It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete so they can avoid their use in new designs, if possible.\n"]
["$I$28",334,"Make/Buy decisions and BOM complete for pilot line build.  Material planning systems in place for pilot line build. ","Purpose: \nTo build qualification test hardware on a Pilot Line using materials planned for LRIP.  The material resource planning system should be validated on the Pilot Line build.  MRLs play a vital role in ensuring that manufacturing processes have been effectively demonstrated in a pilot line environment and that industrial capabilities are reasonably available. In order to accomplish this, the material selection must be complete and stable, and an effective system to manage these materials needs to be established\n \n\nSources of Information:\nThis data should come from engineering and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Make/Buy decisions and Bill of Materials (BOM) complete for pilot line build \n•Material planning systems in place for the pilot line build \n•Risk mitigation plans \n•Manufacturing Maturation Plans (MMPs)\n•BOM documentation\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have all the make or buy decisions been completed for pilot line build?\n2. Is the Bill of Materials completed for pilot line build? \n3. Are material planning systems in place for the pilot line build?\n\n\nAdditional Considerations:\n•What parts do we need to make or buy (Purchasing Plan)? \n•How many of these parts do we need (Capacity Plan)? \n•When must these parts be available (Detailed Manufacturing Schedule)? \n•Have you checked GIDEP alerts for obsolete or DMSMS materials?\n\n\nLessons Learned:\n•The more complex the manufacturing operation the more complex the Materials Requirement Planning (MRP) system.  MRP systems generate two basic outputs, the Purchasing Plan and Schedule that lays out when the purchase orders (POs) should be released and when the purchased items should be received in order to support the production dates. The second output is the Capacity Plan or Production Schedule. \n•The production schedule details the start and completion dates for steps in the production process (routing) to include how many items will be produced in each batch and what is required from the bill of materials to support the fabrication and assembly.  The MRP system should be in place for the Pilot Line build.\n•The Make/Buy decisions and BOM should be complete for Pilot Line build.  Changes to these at this phase in the acquisition cycle could introduce manufacturing risks.  Stable decisions also assist in forecasting, impact analysis, component analysis, and other DMSMS-related activities.   If the underlying information is even slightly off, then you will have problems with your MRP system.  Past issues have been classified by the saying \"garbage in, garbage out.\"  The quality of the following information sources needs to be correct to prevent errors in your MRP system: (1) Master Production System (MPS); (2) Bills of Material (BOM); (3) Inventory and Inventory Status; (4) Lead Times; (5) Production Size; (6) Production Schedule (working times); (7) Quality Data (yield data, scrap, rework and repair, etc.); (8) Safety Stock Levels and Times; and (9) Work-in-Progress Data.\t\n•It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete.\n"]
["$J$28",335,"Make/Buy decisions and BOM complete to support LRIP.  Material planning systems proven on pilot line for LRIP build.","Purpose: \nTo ensure that materials planning has been properly developed to allow for duplicating a product configuration in a cost-effective manner, and that any risks to achieving this goal have been identified and addressed. MRLs play a vital role in ensuring that manufacturing processes have been effectively demonstrated in a pilot line environment and that industrial capabilities are reasonably available. In order to accomplish this, the material selection must be complete and stable, and an effective system to manage these materials needs to be established.  \n\nSources of Information:\nThe majority of the information should be available in the Manufacturing Plan and production floor manufacturing documentation.  Also review the system risk management plans and results.\n\n\nExamples of ‘Objective Evidence’:\n•Make/Buy decisions and Bill of Materials (BOM) are complete to support Low Rate Initial Production (LRIP) \n•Material planning systems proven on the pilot line and are in place for Low Rate Initial Production (LRIP) build \n•Manufacturing Maturation Plans (MMPs)\n•BOM documentation\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Based on the pilot line, are there any changes to the make or buy decisions for initial production (LRIP)?  \n2. Based on pilot line, are there any changes to the Bill of Materials for initial production (LRIP)?  \n2. Have material planning systems (i.e., MRP II, ERP, etc.) been proven on the pilot line and are ready for Low Rate Initial Production (LRIP) build?\n\n\n\nAdditional Considerations:\n•What parts do we need to make or buy (Purchasing Plan)? \n•How many of these parts do we need (Capacity Plan)? \n•When must these parts be available (Detailed Manufacturing Schedule)? \n•Have you continued to check GIDEP alerts for obsolete or DMSMS materials?\n•The more complex the manufacturing operation the more complex the MRP.  MRP systems generate two basic outputs, the Purchasing Plan and Schedule that lays out when the purchase orders (POs) should be released and when the purchased items should be received in order to support the production dates. \n•The second output is the Capacity Plan or Production Schedule. The production schedule details the start and completion dates for steps in the production process (routing) to include how many items will be produced in each batch and what is required from the bill of materials to support the fabrication and assembly.  \n•The MRP system should be in place for the Pilot Line build.\n\n\nLessons Learned:\n•The Make/Buy decisions and BOM should be complete.  Changes to these at this phase in the acquisition cycle could introduce manufacturing risks.  Stable decisions also assist in forecasting, impact analysis, component analysis, and other DMSMS-related activities.   \n•The MRP system should have been uploaded, utilized and proven-out with the Pilot Line build.  If the underlying information is even slightly off, then you will have problems with your MRP system.  Past issues have been classified by the saying \"garbage in, garbage out.\"  \n•The quality of the following information sources needs to be correct to prevent errors in your MRP system: (1) Master Production System (MPS); (2) Bills of Material (BOM); (3) Inventory and Inventory Status; (4) Lead Times; (5) Production Size; (6) Production Schedule (working times); (7) Quality Data (yield data, scrap, rework and repair, etc.); (8) Safety Stock Levels and Times; and (9) Work-in-Progress Data.\t\n•It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete. If no other options are available, a life-time buy of those parts/materials should be considered.\n"]
["$K$28",336,"Make/Buy decisions and BOM complete to support FRP.  Material planning systems proven in LRIP and sufficient for FRP.","Purpose: \nTo ensure that an adequate manufacturing approach has been demonstrated in a low rate production environment and that adequate planning is available for cost-effective production at high rates.    \n\n\nSources of Information:\nThe majority of the information should be available in the Manufacturing Plan and production floor manufacturing documentation.  \n\n\nExamples of ‘Objective Evidence’:\n•Make/Buy decisions documented for FRP \n•BOM documentation for FRP \n•Evidence that the material planning system was proven in LRIP and sufficient for FRP\n•Manufacturing Maturation Plans (MMPs)\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Based on initial production, are there any changes to the make or buy decisions for Full Rate Production (FRP)?\n2. Based on initial production, are there any changes to the Bill of Materials for Full Rate Production (FRP)?\n3. Have material planning systems (i.e., MRP II, ERP, etc.) been proven in LRIP?\n4. Is the material planning systems (i.e., MRP II, ERP, etc.) sufficient for Full Rate Production (FRP)?\n\n\n\nAdditional Considerations: \n• What parts do we need to make or buy (Purchasing Plan)? \n• How many of these parts do we need (Capacity Plan)? \n• When must these parts be available (Detailed Manufacturing Schedule)?\n• Have you continued to check GIDEP alerts for obsolete or DMSMS materials?\n• The more complex the manufacturing operation the more complex the MRP.  MRP systems generate two basic outputs, the Purchasing Plan and Schedule that lays out when the purchase orders (POs) should be released and when the purchased items should be received in order to support the production dates. \n• The second output is the Capacity Plan or Production Schedule. The production schedule details the start and completion dates for steps in the production process (routing) to include how many items will be produced in each batch and what is required from the bill of materials to support the fabrication and assembly.\n\nLessons Learned:\n• Increasing the production rate capability to FRP requirements effects many aspects of the manufacturing operations.  Proactive planning is critical to ensure that efficient and economical production at these higher rates can be achieved.  \n• The Make/Buy decisions and BOM must be stable for effective transition into a high rate production environment.  Changes to these at this phase in the acquisition cycle could introduce manufacturing risks.  Stable decisions also assist in forecasting, impact analysis, component analysis, and other DMSMS-related activities.   \n• The MRP system should have been uploaded, utilized and proven-out with the LRIP build.  If the underlying information is even slightly off, then you will have problems with your MRP system.  Past issues have been classified by the saying \"garbage in, garbage out.\"  \n• The quality of the following information sources needs to be correct to prevent errors in your MRP system: (1) Master Production System (MPS); (2) Bills of Material (BOM); (3) Inventory and Inventory Status; (4) Lead Times; (5) Production Size; (6) Production Schedule (working times); (7) Quality Data (yield data, scrap, rework and repair, etc.); (8) Safety Stock Levels and Times; and (9) Work-in-Progress Data.\t\n• It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete. If no other options are available, a life-time buy of those parts/materials should be considered."]
["$L$28",337,"Material planning systems validated on FRP build. ","Purpose: \nTo ensure that the material selections are complete and stable, and that an adequate materials planning system is in place and proven to allow for continued cost-effective production at high rates.\n\n\nSources of Information:\nThis data should come from program, engineering and manufacturing departments.\n\n\nExamples of ‘Objective Evidence’:\n•Material planning system \n•Evidence that the material planning system has been validated on the FRP build \n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have the material planning systems (i.e., MRP II, ERP, etc.) been validated by FRP?\n\n\nAdditional Considerations:\n• Have you continued to check GIDEP alerts for obsolete or DMSMS materials?\n\n\nLessons Learned:\n• It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete. If no other options are available, a life-time buy of those parts/materials should be considered."]
["MilestoneA",338,null,"A successful Milestone A decision initiates the Technology Development phase. Per DoD Instruction 5000.02, this phase matures technologies, determines the appropriate set of technologies to be integrated into a full system, conducts competitive prototyping of system elements, refines requirements, and develops the functional and allocated baselines of the end-item system configuration. If a platform or system depends on specific technologies to meet system operational threshold requirements in development, production, operation, and sustainment, and if the technology or its application is either new or novel, then that technology is considered a critical or enabling technology. If there are any critical technology elements, they are to be evaluated during the Technology Development phase to assess technology maturity. Technology Development is a focused effort to mature, prototype, and demonstrate technologies in a relevant environment. This results in a preferred system concept that achieves a level suitable for low risk entry into Engineering and Manufacturing Development. This can best result from a close collaboration between the science and technology community, the user, and the engineering community.\nThis phase develops and demonstrates prototype designs to reduce technical risk, validate designs, validate cost estimates, evaluate manufacturing processes, and refine requirements. Based on refined requirements and demonstrated prototype designs, Integrated Systems Design of the end-item system can be initiated. When approved in the Technology Development Strategy, the Integrated Systems Design may be conducted prior to Milestone B, to include full functional analysis of the end-item system, definition of the functional baseline, following the System Functional Review; functional allocation to system configuration items; and establishment of the system allocated baseline following the Preliminary Design Review. Additionally, the Technology Development Phase efforts ensure the level of expertise required to operate and maintain the product is consistent with the force structure. Technology development is an iterative process of maturing technologies and refining user performance parameters to accommodate those technologies that do not sufficiently mature (requirements trades). The Initial Capabilities Document, the Technology Development Strategy, and the maturing draft Capability Development Document (CDD) guide the efforts of this phase, leading to the approved CDD.\nCompetitive prototyping and effective employment of systems engineering, applied in accordance with a well-structured Systems Engineering Plan, and monitored with meaningful technical reviews, will reduce program risk, identify potential management issues in a timely manner and support key program decisions.\n4.3.2.1. Purpose"]
["MilestoneB",339,null,"     EMD begins at Milestone B, which is normally formal program initiation. This phase is to complete the develop-ment of a system or increment of capability, leveraging design considerations; complete full system integration; develop an affordable and executable manufacturing processes, complete system fabrication, test and evaluation. A key emphasis during EMD is to ensure operational supportability with particular attention to minimizing the logistics footprint. \n     The purposes of EMD are to: Develop a system or increment of capability, Reduce integration and manufacturing risk, Design-in critical supportability aspects to ensure materiel availability with particular attention to reducing the logistics footprint, Integrate hardware, software, and human systems, Design for producibility, Ensure affordability and protection of critical program information, and Demonstrate system integration, interoperability, supportability, safety, and utility.\n     In EMD, the program, the system architecture, and system elements down to the configuration item (hardware and software) level are defined based upon the mature technology suite selected and integrated during Materiel Solution Analysis and Technology Development. During EMD, system design requirements are allocated down to the major subsystem level, and are refined as a result of developmental and operational tests, and iterative systems engineering analyses. The support concept and strategy are refined with detailed design-to requirements determined for the product support package elements.\n     Effective employment of systems engineering, applied in accordance with a well structured Systems Engineering Plan, and monitored with meaningful metrics and technical reviews, will reduce program risk and identify potential management issues in a timely manner.\n     EMD consists of two major, sequential efforts: Integrated System Design and System Capability and Manufactur-ing Process Demonstration. The EMD systems engineering work effort typically completes the Integrated System Design (including all initial technical reviews not previously completed in Technology Development, and technical reviews intended to occur during EMD) and a System Capability and Manufacturing Process Demonstration. EMD begins when the program manager has an allocated baseline for the system or increment of capability but has not developed or integrated the end item components and subsystems into a fully operational and supportable system. EMD systems engineering work also completes any remaining initial systems design activities not finished during the Technology Development phase (i.e., System Requirements Review, System Functional Review, or Preliminary Design Review).\n     The primary purpose of systems engineering in EMD is to reduce system-level risk. Through the conduct of systems engineering, the efforts in this phase integrate components and subsystems, and complete the detailed design to meet performance requirements with a producible and sustainable design, and reduce system level risk. EMD typically includes the demonstration of production prototype articles or engineering development models. When the necessary industrial capabilities are available, the system satisfies approved requirements, and the system meets or exceeds exit criteria and Milestone C entrance requirements, the EMD effort may end. Key to the systems engineering in the EMD phase is acceptable performance in integrated test, developmental evaluation and operational assessments, and the use of modeling and simulation in test and evaluation and the demonstration of satisfactory system integration."]
["MilestoneC",340,null,"     The Production and Deployment Phase commences at Milestone C. During the Production and Deployment phase, the system should achieve operational capability that satisfies mission needs.\n     Two work efforts, separated by the Full-Rate Production Decision Review, comprise the Production and Deployment Phase: Low-Rate Initial Production and Full-Rate Production and Deployment.\n     Effective employment of systems engineering, applied in accordance with a well structured Systems Engineering Plan, and monitored with meaningful technical reviews, will reduce program risk and identify potential management issues in a timely manner."]
["MilestoneFRP",341,null,"     The program manager should use existing manufacturing processes whenever possible. When the design requires new manufacturing capabilities, the program manager needs to consider process flexibility (e.g., rate and configuration insensitivity). Full-Rate Production of a system necessitates a stable design, proven manufacturing processes, and available or programmed production facilities and equipment (see the Production Readiness Review for additional information).\n     Manufacturing readiness is a measure of producibility. It is the ability to harness the manufacturing, production, quality assurance, and industrial functions to achieve an operational capability that satisfies mission needs in the quantity and quality needed by the war-fighter to carry out assigned missions at the \"best value\" as measured by the warfighter. Best value refers to increased performance as well as reduced cost for developing, producing, acquiring, and operating systems throughout their life cycle. Manufacturing readiness begins before, continues during the development of systems, and even after a system has been in the field for a number of years.\n     Assessment of Manufacturing Readiness involves continuous assessments performed throughout the acquisition process and in conjunction with the systems engineering technical reviews that specifically measure manufacturing readiness from the Materiel Solution Analysis phase all the way through the Production and Deployment phase. The assessments focus on identifying and mitigating manufacturing risk and ensuring a program or technology is ready to move forward based on a manufacturing perspective."]

{"sheet":"Criteria","index":4,"n_rows":12,"n_cols":8,"rows_kept":12,"cut":false,"hidden":true,"merged":[[3,2,3,3]]}
["Make use of named ranges and the vlookup function."]
["Get and Set values using subs and functions in modDatabase."]
[]
["Set these",null,"Get these",null,null,null,null,"Used to set control sources"]
["JTRecordRef",null,"JTMatrixText","JTReferenceText","JTLink1Text","JTLink1Address",null,"JTRefRow"]
["$F$24",null,{"$f":"IF(VLOOKUP($A6,JTDatabase,3,FALSE)=0,\"\",VLOOKUP($A6,JTDatabase,3,FALSE))","v":"Workforce skill set and production workforce requirements (technical and operational) for the preferred materiel solution identified and considered in the AoA. Workforce training and development requirements to close skill set gaps defined. Availability of workforce for TMRR Phase determined."},{"$f":"IF(VLOOKUP($A6,JTDatabase,4,FALSE)=0,\"\",VLOOKUP($A6,JTDatabase,4,FALSE))","v":"Purpose:\nTo perform a top level assessment for the program looking for areas of risk driven by workforce skills requirements. Where risks are identified, appropriate risk reduction efforts must be put into place in the TMRR Phase.  In addition, cost should be reflected in both the estimates to execute the TMRR phase and overall lifecycle cost projections. The program must ensure the manufacturing workforce risk is assessed for each proposed alternative to make informed decisions on total risk of achieving program cost, schedule, and performance objectives.\n\n\nSources of Information: \nThe Quality Manager, Design Engineer, Manufacturing Engineer, and Resource Management will be good places to get an understanding of the skills available at their facility.\n\n\nExamples of ‘Objective Evidence’:\n•  Analysis of Alternatives (AoA)\n•  Necessary manufacturing workforce skills identified and documented – employee & personnel training records\n•  Production workforce requirements (technical and operational) evaluated – employee & personnel training records\n•  Availability of the process development workforce for the Technology Maturation and Risk Reduction Phase documented\n\n\n\nQuestions (MRL Criteria in the form of a question):\n1. Have workforce skill set and production workforce requirements (technical and operational) for the preferred materiel solution been identified?\n2. Have workforce skill set and production workforce requirements (technical and operational) for the preferred materiel solution been considered in the AoA?\n3. Have workforce training and development requirements to close skill set gaps been defined?\n4. Has the availability of the workforce for TMRR Phase been determined?\n\n\n\nAdditional Considerations:\n•  Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?\n\n\nLessons Learned: \n•  When you are looking at skills, you must account for all resources required to build, test/inspect and support labor (e.g. Manufacturing Engineers) for the units. \n•  If there is a new process, or tolerances are more difficult, there exists a stronger probability of new or critical skills required that may be difficult to obtain.\n"},{"$f":"IF(VLOOKUP($A6,JTDatabase,5,FALSE)=0,\"\",VLOOKUP($A6,JTDatabase,5,FALSE))","v":""},{"$f":"IF(VLOOKUP($A6,JTDatabase,6,FALSE)=0,\"\",VLOOKUP($A6,JTDatabase,6,FALSE))","v":""},null,{"$f":"IF(VLOOKUP($A6,JTDatabase,2,FALSE)=0,\"\",VLOOKUP($A6,JTDatabase,2,FALSE))","v":283}]
[]
["JTTechnologyAreaRef",null,"JTTechnologyArea"]
["$B$24",null,{"$f":"IF(VLOOKUP($A9,JTDatabase,3,FALSE)=0,\"\",VLOOKUP($A9,JTDatabase,3,FALSE))","v":"G.1 - Mfg Workforce (Engineering & Production)"}]
[]
["JTMaturityLevelRef",null,"JTMaturityLevel"]
["$F$5",null,{"$f":"IF(VLOOKUP($A12,JTDatabase,3,FALSE)=0,\"\",VLOOKUP($A12,JTDatabase,3,FALSE))","v":"MRL 4"}]

{"sheet":"MRL Users Guide","index":5,"n_rows":28,"n_cols":12,"rows_kept":28,"cut":false,"hidden":false,"merged":[[3,0,3,1],[1,0,1,11],[2,0,2,1],[2,2,2,4],[2,6,2,7],[2,8,2,9],[24,0,25,0],[26,0,27,0],[6,0,7,0],[8,0,9,0],[10,0,12,0],[13,0,16,0],[17,0,19,0],[20,0,22,0]]}
["Version 2020\n","User's Guide v.2020"]
["DoD Manufacturing Readiness Levels (MRLs)"]
["Acquisition Phase",null,"Pre-Materiel Development Decision (Pre-MDD)",null,null,"Materiel Solution Analysis (MSA)  ","Technology Maturation and Risk Reduction (TMRR)",null,"Engineering & Mfg Development (EMD)",null,"Low-Rate Initial Production (LRIP)","Full-Rate Production (FRP)"]
["Technical Reviews",null,null,null,null,"ASR","SRR/SFR","PDR","CDR","PRR/SVR","PCA"]
["Thread","Sub-Thread","MRL 1","MRL 2","MRL 3","MRL 4","MRL 5","MRL 6","MRL 7","MRL 8","MRL 9","MRL 10"]
[null,"Technology\nMaturity ","Should be assessed at TRL 1.","Should be assessed at TRL 2.","Should be assessed at TRL 3.","Should be assessed at TRL 4.","Should be assessed at TRL 5.","Should be assessed at TRL 6.","Should be assessed at TRL 7","Should be assessed at TRL 7 or 8.","Should be assessed at TRL 8 or 9.","Should be assessed at TRL 9."]
["A- Technology and Industrial Base","A.1 - Industrial base","Global trends in emerging industrial ...","Potential industrial base capability gaps …","Industrial base capabilities for potential …","Industrial base capabilities surveyed for …","Industrial base capabilities assessment initiated ...","Industrial base capabilities assessment for ...","Industrial capability to support production …","Industrial base capability assessment for ...","Industrial capability assessment for FRP ...","Industrial capability supports FRP and ..."]
[null,"A.2 - Manufacturing Technology Development","Global trends in manufacturing science ...","Potential manufacturing science and technology ...","Manufacturing technology requirements identified to ...","Manufacturing technology development initiatives defined …","Required manufacturing technology development efforts ...","Manufacturing technology efforts continuing.  ...","Manufacturing technology efforts continuing.  ...","Primary manufacturing technology efforts concluding. ...","Manufacturing technology process improvements efforts ...","Manufacturing technology continuous process improvements ..."]
["B - Design   ","B.1 - Producibility Program","Hypotheses developed for cause-effect relationships ...","Studies performed to test hypotheses …","System concept elements evaluated for …","Initial producibility and manufacturability assessments ...","Producibility and manufacturability assessments of ...","Producibility assessments and producibility trade ...","Detailed producibility trade studies using ...","Producibility improvements implemented on system. ...","Prior producibility improvements analyzed for ...","Design producibility improvements demonstrated in ..."]
[null,"B.2 - Design Maturity","Current capability deficiencies and gaps ...","Analyses performed to evaluate the ...","High level performance, lifecycle, and ...","Form, fit, and function constraints, ...","Lower level performance requirements sufficient ...","System allocated baseline established.  ...","Product design and features are …","Detailed design of product features ...","Major product design features and ...","Product design is stable.  ..."]
["C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Hypotheses developed regarding technology impact ...","Cost model approach defined.","Manufacturing cost estimates for system ...","Cost estimates refined based on ...","Prototype components produced in a ...","Cost model updated with design …","Cost model updated with the ...","Cost model updated with results ...","FRP cost model updated with ...","Cost model validated against actual ..."]
[null,"C.2 - Cost Analysis","Initial manufacturing and quality costs …","Potential manufacturing and quality cost ...","Analyses conducted to refine manufacturing ...","Producibility and lifecycle cost risks ...","Costs analyzed using prototype component ...","Costs analyzed using prototype system/sub-system ...","Manufacturing costs rolled up to ...","Costs analyzed using pilot line ...","LRIP cost goals met and ...","FRP cost goals met.  ..."]
[null,"C.3 - Manufacturing Investment Budget","Potential manufacturing investment strategy developed.","Program/projects have reasonable budget estimates …","Program/projects have reasonable budget estimates ...","Manufacturing technology budget initiatives developed …","Program has updated budget estimate ...","Program has reasonable budget estimate …","Program has updated budget estimate ...","Program has reasonable budget estimate ...","Materials controlled to specifications in ...","Production budgets sufficient for production ..."]
["D - Materials (Raw Materials, Components, Sub-assemblies and Sub-systems)","D.1 - Maturity","New material properties and characteristics …","Potential effects of new material ...","Effects of new material properties ...","New materials and components for ...","Materials manufactured or produced in ...","Material maturity verified through technology ...","Material maturity sufficient for pilot ...","Materials proven and validated during …","Material is controlled to specification ...","Materials controlled to specifications in ..."]
[null,"D.2 - Availability","Global trends for material availability, …","Material availability, obsolescence, and DMSMS ...","Material availability, obsolescence, and DMSMS …","Projected lead times identified for ...","Availability risks and issues addressed ...","Availability risks and issues addressed ...","Availability risks and issues addressed ...","Availability risks and issues managed ...","Long lead procurement initiated for ...","All material availability risks and …"]
[null,"D.3 - Supply Chain Management","Global trends for supply chain ...","Potential supply chain capability and …","Supply chain capability and capacity ...","Survey of potential supply chain ...","Potential supply chain sources identified ...","Lifecycle Supply Chain requirements updated. ...","Effective supply chain management processes ...","Assessment of critical second and ...","Long term agreements in place ...","Supply chain proven and supports ..."]
[null,"D.4 - Special Handling (i.e. GFP, shelf life, security, hazardous materials, storage environment, ESH, etc.) ","Hazardous materials identified and safety ...","Initial evaluation of potential regulatory ...","ESH compliance risk identified. List ...","ESH compliance risk mitigated in ...","ESH requirements and special handling ...","ESH requirements addressed and documented. ...","ESH compliance demonstrated in production ...","ESH compliance demonstrated in pilot ...","ESH compliance demonstrated in LRIP. ...","ESH compliance demonstrated in FRP. ..."]
["E - -Process Capability & Control","E.1 - Modeling & Simulation (Product & Process)","Modeling and simulation approaches/tools identified …","Modeling and simulation in development …","Manufacturing and quality gaps for ...","Production modeling and simulation tools ...","Initial modeling & simulations (product ...","Initial modeling & simulations developed ...","Modeling & simulations used to ...","Modeling & simulations verified by ...","Modeling & simulations verified by ...","Modeling & simulations verified by ..."]
[null,"E.2 - Manufacturing Process Maturity","Hypotheses developed regarding cause-effect relationships ...","Studies performed to test hypotheses …","Initial estimates of yields and ...","Maturity of critical processes for ...","Process maturity assessed on similar ...","Manufacturing processes demonstrated in production ...","Manufacturing processes demonstrated in a ...","Manufacturing processes for LRIP verified ...","Manufacturing processes are stable, adequately ...","Manufacturing processes are stable, adequately ..."]
[null,"E.3 - Process Yields and Rates","Hypotheses developed regarding future state ...","Studies performed to test hypotheses ...","Initial estimates of yields and ...","Yield and rate assessments on …","Target yields and rates established ...","Yields and rates from production …","Yields and rates from production ...","Pilot line targets achieved. Yields ...","LRIP yield and rate targets ...","FRP yield and rate targets …"]
["F - Quality","F.1 - Quality Management.","Quality management considerations surveyed and ...","Quality management needs assessed, analyzed, ...","Quality management requirements for system ...","Quality strategy for the preferred ...","Quality strategy updated to reflect …","Initial Quality Plan and QMS ...","Quality targets established. QMS elements ...","Program-specific Quality Program Plan established. ...","Quality targets verified on LRIP ...","Quality targets verified on FRP ..."]
[null,"F.2 - Product Quality","Quality metrology state of the …","Studies performed to test hypotheses ...","System concept elements evaluated for ...","Product quality requirements and the …","Roles and responsibilities identified for ...","KC management approach defined.  ...","Quality data from the production ...","KCs managed. Measurement procedures and ...","Data from LRIP demonstrates production ...","KCs controlled at rate. Results ..."]
[null,"F.3 - Supplier Quality/ Management","Supplier quality and quality management ...","Initial supplier quality and quality ...","Supplier quality and quality management ...","Potential supplier quality capabilities, risks, ...","Supply base quality capabilities and ...","Supply base quality improvement initiatives ...","Key supplier QMSs meet appropriate ...","Supplier program-specific QMSs adequate. Supplier ...","Supplier quality management of KCs ...","Supplier quality data reflects adequate ..."]
["G - Mfg Workforce (Engineering & Production)","G.1 - Mfg Workforce (Engineering & Production)","Workforce skill sets to support …","Workforce skill sets to support ...","Workforce skill set requirements for ...","Workforce skill set and production ...","Skill sets identified and plans ...","Manufacturing workforce skills available for ...","Manufacturing workforce resource requirements identified ...","Manufacturing workforce resource requirements identified ...","LRIP personnel requirements met. Plan ...","FRP personnel requirements met.  ..."]
["H - Facilities","H.1 - Tooling/STE/SIE","State of the art tooling, ...","Potential tooling, STE, and SIE ...","Tooling, STE, and SIE requirements ...","Tooling/STE/SIE requirements for the preferred ...","Tooling and STE/SIE requirements identified …","Prototype tooling and STE/SIE concepts ...","Design and development efforts for ...","Tooling, test and inspection equipment ...","All tooling, test and inspection ...","Proven tooling, test and inspection ..."]
[null,"H.2 - Facilities","Current facility capabilities and capacity ...","Potential facility capabilities and capacity …","Facility capabilities and capacity requirements ...","Capability and availability of manufacturing …","Manufacturing facilities identified and plans ...","Manufacturing facilities identified and plans ...","Manufacturing facilities identified and plans ...","Pilot line facilities demonstrated. Manufacturing ...","Manufacturing facilities in place and ...","Production facilities in place and ..."]
["I - Mfg Management","I.1 - Mfg Planning & Scheduling","Manufacturing management considerations surveyed and …","Manufacturing management needs assessed, analyzed ...","Manufacturing management requirements for system ...","Manufacturing strategy for the preferred ...","Mfg. strategy refined based upon …","Initial manufacturing approach developed.  ...","Initial Manufacturing Plan developed and ...","Manufacturing Plan updated for LRIP. ...","Manufacturing plan updated for FRP. ...","All manufacturing risks and issues ..."]
[null,"I.2 - Materials Planning  ","Materials planning state of the ...","Initial availability, lead time, handling ...","Materials and components list for ...","Materials and components list with ...","Make/buy evaluations initiated and include ...","Most material make/buy decisions complete, ...","Make/Buy decisions and BOM complete ...","Make/Buy decisions and BOM complete ...","Make/Buy decisions and BOM complete ...","Material planning systems validated on ..."]

{"sheet":"Definitions","index":6,"n_rows":109,"n_cols":2,"rows_kept":109,"cut":false,"hidden":false,"merged":[[0,0,0,1]]}
["Definitons"]
["Acquisition Strategy: "," A business and technical management approach designed to achieve program objectives within the resource constraints imposed. It is the framework for planning, directing, contracting for, and managing a program. It provides a master schedule for research, development, test, production, fielding, modification, post-production management, and other activities essential for program success. The acquisition strategy is the basis for formulating functional plans and strategies (e.g., Test and Evaluation Master Plan (TEMP), Systems Engineering Plan (SEP), Manufacturing Management Plan (MMP), Quality Assurance Plan (QAP), etc.). Once approved by the MDA, the Acquisition Strategy provides a basis for more detailed planning. "]
["Advanced Technology Development: ","A demonstration of the maturity and potential of advanced technologies for enhanced military operational capability or cost effectiveness."]
["Affordability:  ","The degree to which the life cycle cost of an acquisition program is in consonance with the long-range investment plans of the customer."]
["Alternative Systems Review (ASR): ","A multi-disciplined technical review to ensure that requirements agree with the customers’ needs and expectations and that the system under review can proceed into the Technology Maturation and Risk Reduction (TMRR) phase. The ASR should be completed prior to Milestone A. "]
["Analysis of Alternatives (AoA): ","The AoA assesses potential materiel solutions to satisfy the capability need documented in the approved Initial Capabilities Document (ICD). It focuses on identification and analysis of alternatives, measures of effectiveness (MOEs), cost, schedule, concepts of operations, and overall risk, including the sensitivity of each alternative to possible changes in key assumptions or variables. The AoA also assesses critical Technologies (CTs) associated with each proposed materiel solution, including technology maturity, integration risk, manufacturing feasibility, and, where necessary, technology maturation and demonstration needs. The AoA is normally conducted during the Materiel Solution Analysis (MSA) phase of the Defense Acquisition Management System (DAMS), is a key input to the Capability Development Document (CDD), and supports the materiel solution decision at Milestone A. "]
["Applied Research: ","A systematic study to gain knowledge, or understanding, necessary in order to determine the means by which a recognized and specific need may be met. From a manufacturing perspective this level is characterized by assessing the application of the manufacturing capabilities, capacities, or materials needed to meet the specific need.   Applied research translates basic research into solutions for broadly defined military needs.  Typically this level of readiness includes identification, paper studies and analysis of material and process approaches. An understanding of manufacturing feasibility and risk should be emerging at the point when a clear definition of how the technology fits into a military product with a good understanding of the risk at the completion of the product definition."]
["Basic Research: ","A systematic study directed toward greater knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications toward processes or products in mind. It includes all scientific study and experimentation directed toward increasing fundamental knowledge and understanding in those fields of the physical, engineering, environmental, and life sciences related to long-term national security needs. It is farsighted, potentially high-payoff research that provides the basis for technological progress. "]
["Bill of Materials (BOM):  ","A list of the subordinate parts (electronic, electrical, mechanical) in an assembly (e.g., an SRU/SRA or a subsystem assembly). An indentured BOM depicts the top-down breakout relationship of parts to the next higher assembly components (from system to box to board).  At this point, the BOM should be complete to allow for purchasing of components to be used in the Pilot Line build."]
["Capability Development Document (CDD):  ","The Capability Development Document (CDD) identifies operational performance attributes of the proposed system. The CDD is system specific and applies to a single increment of capability in an evolutionary acquisition program. Each increment of a program will either have its own CDD or a separate annex on a master CDD. Key Performance Parameters (KPP) are introduced in the CDD. Cost will be included in the CDD as life-cycle cost or, if available, total ownership costs. The format for the CDD is found at Appendix A to Enclosure F of the Manual for the Operation of the Joint Capabilities Integration and Development System, updated January 2012.   The CDD is prepared during the Technology Development phase to guide the Engineering & Manufacturing Development phase by defining measurable and testable capabilities. The CDD supports the Milestone B decision and must be validated and approved before MS B."]
["Capability Production Document (CPD): ","The Capability Production Document (CPD) identifies production attributes for a single increment of a program. The CPD is rewritten for each increment in an evolutionary acquisition program. Key Performance Parameters (KPP) and performance attributes are refined in the CPD. Cost and engineering estimates will also be refined in the CPD and will be presented as life-cycle cost.  The CPD is prepared during the Engineering & Manufacturing Development phase to guide the Production and Deployment phase and is used to measure the contractor’s delivery."]
["Continuous Improvement:  ","An ongoing effort to improve products, services or processes. These efforts can seek “incremental” improvement over time or “breakthrough” improvement all at once.  Among the most widely used tools for continuous improvement is a four-step quality model—the plan-do-check-act (PDCA) cycle, also known as Deming Cycle or Shewhart Cycle, and Lean/Six Sigma models for improvement."]
["Counterfeit Part: ","An unauthorized copy or substitute part that has been identified, marked, and/or altered by a source other than the part's legally authorized source without legal right or authority to do so or one whose material, performance, or characteristics are knowingly misrepresented by a supplier in the supply chain.  "]
["Critical Application Item:  ","An item essential to system performance or operation or the preservation of life or safety of operating personnel. (Source:  Defense Logistics Agency Regulation 3200.3)"]
["Critical Characteristic:  ","A characteristic that is likely, if defective, to create or increase a hazard to human safety, or to result in failure of a system to perform a required function. (Source:  DoD-STD-2101)"]
["Critical Design Review (CDR):  ","A CDR of a defined initial product baseline is conducted to support development of the prototype system, subsystems, assemblies, or components.  The CDR should ensure that the system under review can proceed into system fabrication, demonstration, and test, and can meet the stated performance requirements within cost (program budget), schedule (program schedule), risk, and other system constraints. Results of the manufacturing assessments need to be used specifically to determine whether the system under review can meet cost, schedule, and performance objectives.  "]
["Critical Manufacturing Process:  ","A process that creates or substantially affects a key or critical characteristic."]
["Critical Parts:","Materials that represent high availability, safety, mission performance, or quality risks to normal production rates or to maximum intended production rates.  They include semi-processed or finished parts with potential risk to a program due to problems such as long lead-time, manufacturing maturity, historic quality problems, foreign source, or diminishing domestic manufacturing source."]
["Critical Safety Item"," A part or assembly whose failure or malfunction could cause a catastrophic or critical failure resulting in loss of life, permanent disability or major injury, loss of a system, or significant equipment damage.  (Source  Defense Acquisition Guide)"]
["Critical Supplier","  A contractor whose performance could seriously jeopardize the successful achievement of a program’s cost, schedule, technical, or supportability requirements if not satisfactorily managed (e.g., a sole source supplier or supplier of critical parts, strategic and critical materials, or unique or special processes.)"]
["Critical Technology (CT)","  Technologies that consist of  (a) arrays of design and manufacturing know-how (including technical data (TD)); (b) keystone manufacturing, inspection, and test equipment; (c) keystone materials; and (d) goods accompanied by sophisticated operation, application, or maintenance know-how that would make a significant contribution to the military potential of any country or combination of countries and that may prove detrimental to the security of the United States. "]
["Criticality"," Characteristics that make a product or service difficult to replace if disrupted. Also, a relative measure of the consequences of a failure mode and its frequency of occurrence."]
["Design for Assembly (DFA) /Design for Manufacturing (DFM)"," Systematic analyses of the design of product, assembly, or subassembly to reduce product cost by simplifying its design, assembly, and manufacturing without impacting performance.  DFA focuses on products designed with ease of assembly in mind, such as reduction in number of parts or inclusion of self-orienting features.  DFM focuses on producibility and lowering costs for individual parts."]
["Diminishing Manufacturing Sources and Material Shortages (DMSMS)","The loss or impending loss of manufacturers of items or suppliers of items or the loss or impending loss of raw materials used to manufacture items of supply. Such losses may cause material shortages that endanger a weapon system’s or equipment’s development, production, or post-production support capability.  (DoDM 4140.01, Volume 3)"]
["Define, Measure, Analyze, Improve and Control (DMAIC)"," DMAIC refers to a data-driven improvement cycle used for improving, optimizing and stabilizing business processes and designs.  All of the DMAIC process steps are required and always proceed in this order. The DMAIC improvement cycle is the core tool used to drive Six Sigma projects. DMAIC is not exclusive to Six Sigma and can be used as the framework for other improvement applications."]
["Failure Modes and Effects Criticality Analysis (FMECA)"," Procedure by which each potential failure mode is analyzed to determine its effects on the system and then classified according to its severity. It further attempts to identify all single points of failure; that is, those points where failure of the component can cause failure of the entire system."]
["Failure Reporting And Corrective Action System (FRACAS)"," A system, sometimes carried out using software, that provides a process for reporting, classifying, analyzing failures, and planning corrective actions in response to those failures. It is typically used in an industrial environment to collect data, record and analyze system failures. A FRACAS system may attempt to manage multiple failure reports that are recorded by numerous individuals in different ways. FRACAS produces a history of failure and corrective actions."]
["First Article Inspection (FAI)"," A complete, independent, and documented physical and functional inspection to verify that prescribed production methods have produced a conforming item as specified by contract, engineering drawings, planning, purchase order, engineering specifications, and/or other applicable design documents.  Independent inspections refer to the inspection being performed by someone other than the person building the product. (Source   AS9102)"]
["First Article Testing (FAT)"," Testing and evaluating the first article for conformance with specified contract requirements before or in the initial stage of production.  (Source   Federal Acquisition Regulations, Subpart 2.101)"]
["Fragility"," Characteristics that make small deviations in the status quo likely to have substantial effects on an industry / supplier."]
["Full Rate Production (FRP)"," Production of economic quantities following stabilization of the system design and prove-out of the production process."]
["Full-Rate Production /Deployment of Production & Deployment Phase","  The second effort of the Production and Deployment (P&D) phase defined and established by DoDI 5000.2. This effort follows a successful Full Rate Production Decision Review (FRPDR). The system is produced at rate production and deployed to the field or fleet. This phase overlaps the Operations and Support (O&S) phase since fielded systems are operated and supported (sustained) while Full Rate Production (FRP) is ongoing."]
["Government Furnished Property","  Property in the possession of, or directly acquired by, the Government and subsequently delivered to the Contractor for performance of a contract. "]
["Government-Industry Data Exchange Program (GIDEP)"," A cooperative activity between government and industry participants seeking to reduce seeking to reduce or eliminate expenditures of resources by sharing technical information essential during research, design, development, production and operational phases of the life cycle of systems, facilities and equipment."]
["Hazardous Material (HAZMAT)","  Any material that due to its chemical, physical, or biological nature that causes safety, public health, or environmental concerns (source   NAS 411) .  Over 83,000 chemicals have been identified under the Toxic Substances Control Act and over 143,000 chemicals have been identified under a similar European Law called Registration, Evaluation, Authorization, and Restrictions of Chemicals (REACH)."]
["Industrial Base Capability Assessment (IBCA)"," The development of the Acquisition Strategy should include results of industrial base capability (public and private) analysis to design, develop, produce, support, and, if appropriate, restart an acquisition program. This includes assessing manufacturing readiness and effective integration of industrial capability considerations into the acquisition process and acquisition programs. "]
["Initial Capabilities Document (ICD)","  Summarizes a capabilities-based assessment (CBA) and recommends materiel or non-materiel approaches or a combination of materiel and non-materiel approaches to satisfy specific capability gaps. It identifies required capabilities and defines capability gap(s) in terms of the joint capability area, the relevant range of military operations; desired effects; time; doctrine, organization, training, materiel, leadership and education, personnel, and facilities (DOTMLPF); and policy implications and constraints. The ICD summarizes the results of DOTMLPF and policy analysis and the DOTMLPF approaches that may deliver the required capability. The outcome of an ICD could be one or more Joint DOTMLPF Change Recommendations (DCRs) or recommendations to pursue materiel solutions."]
["Initial Technical Review (ITR)","  A multi-disciplined technical review held early during the Materiel Solution Analysis (MSA) phase to support a program’s initial Program Objectives Memorandum (POM) submission. The review ensures that a program’s technical baseline is sufficiently rigorous to support a valid cost estimate (with acceptable cost risk), and enable an independent assessment of that estimate by cost, technical, and program management subject matter experts. "]
["Integrated Master Plan (IMP)","  An event-driven plan that documents the significant accomplishments necessary to complete the work and ties each accomplishment to a key program event."]
["Integrated Master Schedule (IMS)","  An integrated and networked multi-layered schedule of program tasks required to complete the work effort captured in a related Integrated Master Plan (IMP). The IMS should include all IMP events and accomplishments and support each accomplishment closure criteria."]
["International Traffic in Arms Regulations (ITAR)","  Issued by the Department of State, control the export of defense-related articles and services, including technical data, ensuring compliance with the Arms Export Control Act (22 U.S.C. 2751 et seq.). The United States Munitions List (USML) identifies defense articles, services, and related technical data that are inherently military in character and could, if exported, jeopardize national security or foreign policy interests of the United States. "]
["Key Characteristic (KC)","  The features of a material or part whose variation has a significant influence on product fit, performance, service life, or manufacturability.  (Source   AS9103)"]
["Key Performance Parameters (KPPs)","  Those attributes or characteristics of a system that are considered critical or essential to the development of an effective military capability and that make a significant contribution to the characteristics of the future joint force. A KPP normally has a threshold representing the minimum acceptable value achievable at low-to-moderate risk, and an objective, representing the desired operational goal but at higher risk in cost, schedule, and performance. KPPs are contained in the Capability Development Document (CDD) and the Capability Production Document (CPD) and are included verbatim in the Acquisition Program Baseline (APB). "]
["Lean Manufacturing","  A management philosophy that uses production tools and principles aimed at creating value for the end customer through the elimination of waste along the entire value stream."]
["Line of Balance","  A production control technique that combines features from a critical path scheduling timeline with a required delivery schedule presenting in graphic form information relating to time and accomplishment of production.  It shows the delivery objective, sequence and duration of all activities required to produce a product, the current status of production items, and, from this data, an assessment showing the relationship of actual component production to schedule."]
["Long Term Agreements"," Business arrangements between two parties (prime & supplier) with long term goals to create program stability to the price and availability of material to reduce risk.   "]
["Low Rate Initial Production (LRIP)","  The first effort of the Production and Deployment (P&D) phase. This effort is intended to result in completion of manufacturing development in order to ensure adequate and efficient manufacturing capability and to produce the minimum quantity necessary to provide production or production-representative articles for Initial Operational Test and Evaluation (IOT&E); establish an initial production base for the system; and permit an orderly increase in the production rate for the system, sufficient to lead to full-rate production (FRP) upon successful completion of operational (and live-fire, where applicable) testing. 2.) At Milestone B, the Milestone Decision Authority (MDA) determines the LRIP quantity for Major Defense Acquisition Programs (MDAPs) and major systems. The LRIP quantity for an MDAP (with rationale for quantities exceeding 10 percent of the total production quantity documented in the acquisition strategy) shall be included in the first Selected Acquisition Report (SAR) after its determination. The LRIP quantity shall not be less than one unit. The Director, Operational Test and Evaluation (DOT&E), following consultation with the program manager (PM), determines the number of production or production- representative test articles required for live-fire test and evaluation (LFT&E) and IOT&E of programs on the Office of the Secretary of Defense (OSD) Operational Test and Evaluation (OT&E) Oversight List. For a system that is not on the Oversight List, the operational test agency (OTA), following consultation with the PM, shall determine the number of test articles required for IOT&E."]
["Major Supplier","  A supplier, distributor, vendor, or firm that furnishes supplies or services to or for the prime contractor whose total costs are a significant portion of the total purchased value for the program.  "]
["Make/Buy Decisions","  That part of a contractor’s written plan for the development or production of an end item that outlines the subsystems, major components, assemblies, subassemblies, and parts the contractor intends to manufacture, test-treat, or assemble (make); and those the contractor intends to purchase from others (buy)."]
["Manufacturability"," The characteristics considered in the design cycle that focus on process capabilities, machine or facility flexibility, and the overall ability to consistently produce at the required level of cost and quality. Associated activities may include some or all of the following  Design for commonality and standardization—uses fewer parts; Design for environmental and safety compliance; Design for multi-use and dual-use applications; Design for modularity and plug compatible interface/integration; Design for flexibility/adaptability or use “robust design”; Utilize reliable processes and materials; Utilize monolithic and determinant assembly; Design for manufacturing and assembly; Achieve production yield."]
["Manufacturing","  The acts, crafts, processes, and methods used to transform tangible inputs such as raw materials, semi-finished goods, and sub-assemblies into goods or services that are suitable for use or have exchange value."]
["Manufacturing Management","  The techniques of planning, organizing, directing, coordinating, and controlling the use of people, money, materials, equipment, research and development, methods and processes, and facilities to manufacture products."]
["Manufacturing Management System","  The integrated collection of the people, processes, policies, information systems, and other tools required to plan, execute, and manage manufacturing operations, including the integration of corresponding supplier activities. "]
["Manufacturing Maturation Plan (MMP)"," A documented risk mitigation plan for an area of manufacturing risk.  The plan describes the actions necessary to achieve the target Manufacturing Readiness Level, including required schedule and resources (e.g. personnel, funding, facilities, and equipment.)"]
["Manufacturing Readiness Levels (MRLs)"," Manufacturing readiness criteria used to assess manufacturing risk, readiness, and producibility.  MRLs provide a common understanding of the relative maturity, identification and mitigation of manufacturing risks associated with manufacturing technologies, products, and processes.  Refer to the DoD MRL Deskbook for additional information."]
["Material Resource Planning (MRP) System"," The MRP system is a production and inventory control software tool to assist in the management of manufacturing processes. Based on sales forecast and backlog, the MRP takes information from three sources to include the Master Production Schedule (MPS), the bill-of-materials (BOM) and inventory status data as inputs to calculate the answer to these questions  (1) What parts do we need to make or buy (Purchasing Plan)? (2) How many of these parts do we need (Capacity Plan)? (3) When must these parts be available (Detailed Manufacturing Schedule)?"]
["Material","  Includes raw material, components, and sub-systems. (Sometimes spelled \"materiel\")"]
["Material Review Board (MRB)","  Representatives of contractor and customer departments necessary to review, evaluate, and determine or recommend disposition of nonconforming material."]
["Measures of Effectiveness (MOE)","A criterion used to assess changes in system behavior, capability, or operational environment that is tied to measuring the attainment of an end state, achievement of an objective, or creation of an effect."]
["Non Recurring Engineering (NRE)"," Refers to the one time cost to research, design, develop and test a new product or product enhancement."]
["Obsolescence","(Please see SD-22 – Diminishing Manufacturing Sources and Material Shortages (DMSMS) Guidebook, Appendix A, for an explanation of obsolescence relationship to DMSMS.)"]
["Physical Configuration Audit (PCA)","  A Physical Configuration Audit (PCA) is used to examine the actual configuration of the CI that is representative of the product configuration in order to verify that the related design documentation matches the design of the deliverable CI. In performance based acquisition environment, the PCA addresses the accuracy of the documentation reflecting the production design. It is also used to validate many of the supporting processes that the contractor uses in the production of the CI. The PCA is also used to verify that any elements of the CI that were redesigned after the completion of the FCA also meet the requirements of the CI's performance specification. In cases where the Government does not plan to control the detail design, it is still essential that the contractor conduct an internal PCA to define the starting point for controlling the production design and to establish a product baseline. Additional PCAs may be accomplished later during CI production if circumstances such as the following apply  (1) The original production line is \"shut down\" for several years and then production is restarted; (2) The production contract for manufacture of a CI with a fairly complex, or difficult-to-manufacture, design is awarded to a new contractor or vendor; (3) This re-auditing in these circumstances is advisable regardless of whether the contractor or the government controls the detail production design."]
["Pilot Line Environment","  A manufacturing environment that incorporates all of the key production realism elements (equipment, personnel skill levels, facilities, materials, components, work instructions, processes, tooling, temperature, cleanliness, lighting, etc.) required to manufacture production configuration items, subsystems or systems that meet design requirements in low rate production."]
["Predictive Indicator","  A set of internal process metrics that can provide quality and delivery forecasts, are actionable, and are indicative of the overall “health” of the organization’s and suppliers’ performance."]
["Preliminary Design Review (PDR)"," the PDR is a multi-disciplined technical review to ensure that a system is ready to proceed into detailed design and can meet stated performance requirements within cost (program budget), schedule (program schedule), risk, and other system constraints. Generally, this review assesses the system preliminary design as captured in performance specifications for each configuration item (CI) in the system (allocated baseline), and ensures that each function in the functional baseline has been allocated to one or more system CIs. Normally conducted during the Technology Development (TD) phase, but may also be conducted early in the Engineering and Manufacturing Development (EMD) phase."]
["Process Capability (Cp/Cpk)","  Is a measure the process capability, not of what it is doing.  Process capability compares the output of an in-control process to the specification limits by using capability indices. The comparison is made by forming the ratio of the spread between the process specifications (the specification \"width\") to the spread of the process values, as measured by 6 process standard deviation units (the process \"width\").  Cpk is short term and assumes that the process is centered."]
["Process Capability Index (Cpk)","  A statistical measure of the ability of a process to produce output within the specification limits. "]
["Producibility"," A design attribute that enables manufacturing to repeatedly fabricate and assemble hardware which satisfies both functional and physical objectives at an optimum cost and within schedule."]
["Product Quality","  The collection of features and characteristics of a product that contribute to its ability to meet given requirements.  Product quality is measured by the degree of conformance to predetermined specifications and standards, and deviations from these standards can lead to poor quality and low reliability.  The identification of the predetermined specifications and standards sets the stage for quality control and then for quality improvement which is aimed at eliminating defects (components and subsystems that are out of conformance), the need for scrap and rework, and hence overall reductions in production costs."]
["Production"," The processes and methods used to transform raw materials, semi-finished goods, sub-assemblies, ideas, information, and knowledge into goods or services that are suitable for use or have exchange value.  Also, a life cycle phase where non-prototype, requirements-compliant products are being manufactured for customer delivery."]
["Production Cost Model"," A framework for incorporating the combined total of raw material and direct labor costs and burden incurred in the production phase of a program including scrap, rework, and repair costs incorporating learning curves over the  production run."]
["Production Line"," An environment that incorporates all verified capabilities required to manufacture production configuration items, subsystems, or systems that meet design requirements utilizing manufacturing processes and procedures that are under control (i.e., PPVs and FAIs/FATs have been satisfactorily completed) and capable of meeting required rate and quantities."]
["Production Process Verification (PPV)","  The use of a representative item from the first production run of a new part or assembly to verify that the production processes, production documentation and tooling are capable of producing parts and assemblies that meet requirements."]
["Production Readiness Review (PRR)"," A formal examination of a program to determine if the design is ready for production, if manufacturing problems have been resolved, and if the organization has adequately planned for the production phase.  PRRs address  management, industrial resources, facilities, design/product engineering, materials and purchased parts, manufacturing, production engineering and planning, quality, reliability, test, software, logistics and training, process improvement, schedule, contracts, cost, and funding."]
["Production Relevant Environment","  An environment with some shop floor production realism present (such as facilities, personnel, tooling, processes, materials etc.). There should be minimum reliance on laboratory resources during this phase. Demonstration in a production relevant environment implies that contractor(s) must demonstrate their ability to meet the cost, schedule, and performance requirements of the EMD Phase based on their production of prototypes. The demonstration must provide the program with confidence that these targets will be achieved, but does not require a production line. Furthermore, there must be an indication of how the contractor(s) intend to achieve the requirements in a production representative and pilot environments."]
["Production Representative Environment","  An environment that has as much production realism as possible, considering the maturity of the design. Production personnel, equipment, processes, and materials that will be present on the pilot line should be used whenever possible. The work instructions and tooling should be of high quality, and the only changes anticipated on these items are associated with design changes downstream that address performance or production rate issues. There should be no reliance on a laboratory environment or personnel."]
["Prototype"," An early sample or model built to test a concept or process. Prototyping serves to provide specifications for a real, working system rather than a theoretical one. Prototypes are not similar to units built on previous programs but is directly tied to your design concept and limitations (e.g. must relate to your design requirements)."]
["Quality Metrics","  A package of charts displaying information about the performance of the design and production systems relating to quality.  This package differs for each program, but the metrics should remain consistent through each phase of the acquisition lifecycle in order to facilitate the identification of quality trends over time.  The metrics package is made up of individual charts, each of which should provide important and unique insight into the design and/or production system’s performance.   "]
["Quality Strategy","  The planned approach to assuring quality throughout the development, production, and support of the product or system.  Items considered during development of a Quality Strategy include design for Quality, inspection plan, statistical sampling plan, defect prevention strategy, acceptance testing, and product acceptance plan."]
["Quality Management System (QMS)","  A formalized system to that organizes activities used to direct and control an organization with regard to quality."]
["Registration, Evaluation, Authorization and Restriction of Chemicals (REACH)"," REACH is a regulation of the European Union, adopted to protect human health and the environment from the risks that can be posed by over 140,000 chemicals.  REACH is comparable to the U.S. Toxic Substances Control (TSC) Act. "]
["Relevant Environment","  Competitive Prototyping and other types of technology maturation need to consider the relevant environment in their planned development applications. The Technology Development Strategy should identify the relevant environment including potential threats against the capabilities desired and the physical environment (altitude, temperature, vibration, electro-magnetic, etc…), within which the materiel solution needs to operate."]
["Risk"," A measure of future uncertainties in achieving program performance goals and objectives within defined cost, schedule, and performance constraints."]
["Risk Management","  The overarching process that encompasses identification, analysis, mitigation planning, mitigation plan implementation, and tracking of program risks. "]
["Robust Requirements Flow Down"," A process that ensures the sub-tier suppliers are formally made aware of the customer engineering standards/specifications and product requirements and there is an official and timely communication channel with the supplier concerning any changes."]
["System Engineering Management Plan (SEMP)"," A document that addresses a contractors overall system engineering management approach. It provides unique insight into the  application of a contractor's standards, capabilities models, configuration management, and toolsets to their organization. "]
["Shelf Life","  The expected length of time in inventory (use) for a system, component, or subassembly."]
["Should Cost model"," A cost model that reflects what it should cost if a program were effectively managed. Leaders should be driving productivity improvement in their programs. “They should be scrutinizing every element of program cost, assessing whether each element can be reduced relative to the year before, challenging learning curves, dissecting overheads and indirect costs, and targeting cost reduction with profit incentive – in short, executing to what the program should cost… "]
["Should Cost Estimate","  An estimate of contract price that reflects reasonably achievable contractor economy and efficiency. It is accomplished by a government team of procurement, contract administration, audit and engineering representatives performing an in-depth cost analysis at the contractor’s and subcontractor’s plants. Its purpose is to develop a realistic price objective for negotiation purposes."]
["Single Source","  Choosing a specific company and bypassing the competition refers to single sourcing. Different suppliers and distributors typically produce and sell similar merchandise. This is advantageous to companies that purchase supplies because they can choose among various companies. However, after a thorough analysis of possible suppliers, company heads, managers or owners may choose to sign with a single company and pass up the opportunity to work with other suppliers. This decision could be based on price or the quality of products."]
["Six Sigma (6s)","  Is a management system and business philosophy that is customer focused, data driven, and process oriented focusing on decisions that target business results (excellence). 6s was developed by the Motorola Corporation in the 1980’s to improve quality, reliability and manufacturing performance.  6s is a methodology for continuous improvement focusing on structured problem solving using the DMAIC process.  6s is also a metric that equates to 3.4 defects per million opportunities, or a 99.99966% yield (accounts for process shifts), or a process capability of either Cp equal to or greater than 2.0, or Cpk equal to or greater than 1.5."]
["Sole Source Contract","  Sole-sourcing contracts vary from single-sourcing contracts, with sole sourcing, there are no options. Companies may search for multiple suppliers and distributors for a product or service, then discover only one vendor is capable of producing the product. "]
["Six Sigma Process"," The application of statistical and mathematical tools to improve the quality of mature and well-controlled manufacturing processes."]
["Special Requirements"," Those requirements which have high risks to being achieved, thus requiring their inclusion in the risk management process.  Special requirements may force you to operate at or near the limit of your organizations manufacturing capability.  (AS 9100C)"]
["Special Storage","  Infers the short- or long-term storing of items. Storage can be accomplished in either temporary or permanent facilities.  Special storage could include the need for refrigeration, humidity controls, etc."]
["Specialized Facilities","  Facilities required to produce a specific product at tolerances, speeds, rates, configurations, materials, environment (e.g., temperatures, clean room, etc.) that have not been done by the contractor proposing to build this alternative."]
["Statistical Process Control","  A methodology utilizing statistical and graphical tools to achieve process stability and improve capability by reducing variability."]
["Strategic and Critical Materials","  Materials not found or produced in the domestic industrial base in sufficient quantities to meet program manufacturing needs.  “Domestic industrial base” refers to the North American industrial base for DoD applications; for commercial applications, the term is defined by the organization.  Strategic and critical materials include any raw material in short supply, limited domestic production, or otherwise posing a risk to a production program."]
["Supplier Capability"," The supply chain has already manufactured and delivered similar product, or has the capability to produce the intended product without major impact to cost, schedule or performance and with the requisite quality achievements."]
["Supplier Survey"," A list of questions/topics for suppliers that can assist in determining their potential technical and financial capabilities. The survey will address the gaps that might be present in the supply chain from a technical, material and process perspective.  "]
["Supply Chain (Manufacturing or Production)"," The linked network of all organizations, resources, activities, and technology associated with the production of finished products, encompassing the tiers of subcontractors contributing their individual stages of manufacturing processing and manufactured components or subassemblies to the final product as assembled and delivered to the customer."]
["Surge"," An increase in the rate of output for a limited period of time to meet sudden and heavy demand (such as in wartime) through multiple work shifts and/or expansion of the production capacity."]
["System Engineering Plan (SEP)"," The purpose of the SEP is to help Government Program Managers develop, communicate, and manage the overall systems engineering (SE) approach that guides all technical activities of the program. The SEP documents key technical risks, processes, resources, metrics, SE products, and completed and scheduled SE activities. The SEP is a living document that should be updated as needed to reflect the program’s evolving SE approach and/or plans and current status.  Manufacturing and Quality Assurance Plans should be incorporated into the SEP. "]
["Takt Time","  The available production time divided by customer demand. Takt time was derived from the German word Taktzeit, translated best as meter, sets the pace for industrial manufacturing lines so that production cycle times can be matched to customer demand rate.  Example of Takt time.  (60 minutes available)/200 units = 18 seconds"]
["Technical Data Package (TDP) Commercial"," For commercial applications, the following definition applies  The collection of data enabling design, production, delivery, and/or maintenance that communicates a customer’s product definition, performance criteria, and method of verification to the source(s) of the deliverable. (Source   AIA/NAS 3500)"]
["Technical Data Package (TDP) DOD"," For DoD applications, the following definition applies   A technical description of an item adequate for supporting an acquisition, production, engineering, and logistics support (e.g., Engineering Data for Provisioning, Training, and Technical Manuals). The description defines the required design configuration or performance requirements, and procedures required to ensure adequacy of item performance. It consists of applicable technical data such as models, drawings, associated lists, specifications, standards, performance requirements, quality assurance provisions, software documentation and packaging details. (Source   MIL-STD-31000)"]
["Technology Readiness Assessment (TRA)"," A statutory requirement for Major Defense Acquisition Programs (MDAPs) and a regulatory information requirement for all other acquisition programs. It is a systematic, metrics-based process that establishes the maturity of critical technology elements. The TRA may be conducted concurrently with other technical reviews such as the Alternative Systems Review (ASR), System Requirements Review (SRR), or the Production Readiness Review (PDR). The Defense Director for Research and Engineering (DDR&E) is required to conduct an independent TRA of MDAPs prior to Milestone B."]
["Variability","  Natural and induced deviations from a target specification caused by controllable and uncontrollable influences."]
["Variability Reduction","  A systematic approach to reducing product and process variability in order to improve cost, schedule, and performance."]

{"sheet":"List of Acronyms","index":7,"n_rows":208,"n_cols":2,"rows_kept":208,"cut":false,"hidden":false,"merged":[[0,0,0,1]]}
["List of Acronyms"]
["A/W","Air Worthiness"]
["ACAT","Acquisition Category"]
["ACO","Administrative Contracting Officer"]
["ACTD","Advanced Concept Technology Demonstration"]
["AFI","Air Force Instruction"]
["AFNWC","Air Force Nuclear Weapons Center"]
["AFPD","Air Force Policy Directive"]
["AFRL","Air Force Research Laboratory"]
["AFSC","Air Force Sustainment Center"]
["AFTC ","Air Force Test Center"]
["ALC","Air Logistics Complex"]
["AoA","Analysis of Alternatives"]
["APDP","Acquisition Professional Development Program"]
["AQS","Advanced Quality System"]
["ARL","Army Research Laboratory"]
["AS","Acquisition Strategy"]
["ASME","American Society of Mechanical Engineers"]
["ASR","Alternative Systems Review "]
["ATD","Advanced Technology Demonstration"]
["BoE","Basis of Estimate"]
["BOM","Bill of Material"]
["CAR","Corrective Action Report (DCMA)"]
["CDD","Capability Development Document"]
["CDR","Critical Design Review"]
["CDRL","Contract Data Requirements List"]
["CJCSI","Chairman of the Joint Chiefs of Staff Instuction"]
["CLC","Corporate Leverage Contracting"]
["CLIN","Contract Line Item Number"]
["CLS","Contractor Logistics Support"]
["CoP","Community of Practice"]
["COTS","Commercial Off-the-Shelf"]
["Cp","Compares what is desired to what is achieved"]
["CPARS","Contractor Performance Assessment Reporting System"]
["CPD","Capability Production Document"]
["CPI","Continuous Process Improvement"]
["Cpk","Process Capability Index"]
["CRAD","Contractor Research and Development"]
["CSI","Critical Safety Item"]
["CT","Critical Technology"]
["DAB","Defense Acquisition Board"]
["DAG","Defense Acquisition Guide"]
["DAL","Data Accession List"]
["DARPA","Defense Advanced Research Projects Administration"]
["DCAA","Defense Contract Audit Agency"]
["DCMA","Defense Contract Management Agency"]
["DFA","Design for Assembly"]
["DFAR","Defense Federal Acquisition Regulation"]
["DFM","Design for Manufacturability"]
["DFx","Design for \"x\""]
["DID","Data Item Description"]
["DMAIC","Define, Measure, Analyze, Improve and Control."]
["DMSMS","Diminishing Manufacturing Sources and Material Shortages"]
["DoD","Department of Defense"]
["DoDD","Department of Defense Directive"]
["DoDI","Department of Defense Instruction"]
["DOE","Design of Experiments"]
["DPAS","Defense Priorities Allocation System"]
["DPMO","Defects per Million Opportunities"]
["DSB","Defense Science Board"]
["DSOR","Depot Source of Repair"]
["DT&E","Developmental Test & Evaluation"]
["DTC","Design to Cost"]
["DUSD(S&T)","Department of the Under Secretary of Defense (Science & Technology)"]
["ECP","Engineering Change Proposal"]
["EMD","Engineering and Manufacturing Development"]
["EPA","Environmental Protection Agency"]
["ESC","Electronic Systems Center"]
["ESH","Environmental, Safety & Health"]
["EVMS","Earned Value Management System "]
["FAA","Federal Aviation Administration"]
["FAI","First Article Inspection"]
["FAR","Federal Acquisition Regulation"]
["FAT","First Article Test"]
["FCA","Functional Configuration Audit"]
["FMEA","Failure Modes and Effects Analysis"]
["FMECA","Failure Modes and Effects Criticality Analysis"]
["FOC","Full Operational Capability"]
["FRACAS","Failure Review and Corrective Action System"]
["FRP","Full Rate Production"]
["GAO","Government Accounting Office"]
["GFE","Government Furnished Equipment"]
["GFP","Government Furnished Property"]
["GIDEP","Government-Industry Data Exchange Program"]
["HASC","House Armed Services Committee"]
["HAZMAT","Hazardous Materials"]
["HMM","Hazardous Material Management"]
["HQA","Hardware Quality Audit"]
["IAW","In Accordance With"]
["IBCA","Industrial Base Capability Assessment"]
["ICD","Initial Capabilities Document"]
["ICE","Independent Cost Estimate"]
["IDIQ","Indefinite Delivery, Indefinite Quantities"]
["ILS","Integrated Logistics Support"]
["IMP","Integrated Master Plan"]
["IMS","Integrated Master Schedule"]
["IOC","Initial Operational Capability"]
["IOT&E","Initial Operational Test & Evaluation"]
["IPT","Integrated Product Team"]
["IRAD","Internal Research and Development"]
["ISO","International Organization for Standards"]
["ITAR","International Traffic in Arms Regulations"]
["ITR","Initial Technical Review"]
["JCIDS","Joint Capabilities Integration Development System"]
["JDMTP","Joint Defense Manufacturing Technology Panel"]
["KC","Key Characteristic"]
["KDP","Key Decision Point"]
["KPP","Key Performance Parameter"]
["LAT","Lot Acceptance Test"]
["LCC","Life Cycle Cost"]
["LCMC","Life Cycle Management Center"]
["LCSE","Life Cycle Systems Engineering"]
["LOB","Line of Balance"]
["LRIP","Low Rate Initial Production"]
["MDA","Milestone Decision Authority"]
["MDAP","Major Defense Acquisition Program"]
["MDD","Materiel Development Decision"]
["ME","Manufacturing Engineer"]
["Mfg","Manufacturing"]
["MGI","Mandatory Government Inspection (normally by DCMA)"]
["MM/PCR","Manufacturing Management/Production Capability Review"]
["MMAS","Material Management Accounting System "]
["MMP","Manufacturing Maturation Plan"]
["MOA","Memorandum of  Agreement"]
["MOE","Measures of Effectiveness "]
["MOU","Memorandum of  Understanding"]
["MRA","Manufacturing Readiness Assessment"]
["MRB","Material Review Board"]
["MRL","Manufacturing Readiness Level"]
["MRO","Maintenance, Repair & Overhaul"]
["MRP","Materials Requirement Planning System"]
["MRP II","Manufacturing Resource Planning System II"]
["MS","Milestone  "]
["MSA","Materiel Solution Analysis"]
["MSE","Manufacturing Systems Engineer"]
["NDAA","National Defense Authorization Act (yearly)"]
["NDI","Non-destructive Inspection"]
["NDIA","National Defense Industrial Association"]
["NEPA","National Environmental Policy Act "]
["NRE","Non Recurring Engineering "]
["NRL","Navy Research Laboratory"]
["O&S","Operations & Support"]
["OEM","Original Equipment Manufacturer"]
["OSD","Office of the Secretary of Defense"]
["OSEP","Organizational Systems Engineering Plan"]
["OSS&E","Operational Safety, Suitability and Effectiveness"]
["PCA","Physical Configuration Audit"]
["PCE","Professional Continuing Education"]
["PCO","Procurement Contracting Officer"]
["PDCA","Plan-Do-Check-Act"]
["PDR","Preliminary Design Review"]
["PEO","Program Executive Officer"]
["PESHE","Programmatic Environmental, Safety, and Health Evaluation "]
["PFMEA","Process Failure Modes and Effects Analysis"]
["PIP","Producibility Improvement Program"]
["PM","Program Manager"]
["PMO","Program Management Office"]
["PMR","Program Management Review"]
["POM","Program Objective Memorandum"]
["POU","Point of Use"]
["PPM","Parts per Million"]
["PPV","Production Process Verification"]
["PQM","Production, Quality and Manufacturing"]
["PRR","Production Readiness Review"]
["PSR","Program Support Review"]
["QA","Quality Assurance"]
["QFD","Quality Function Deployment"]
["QMS","Quality Management System"]
["QP","Quality Plan"]
["R&R (Gauge)","Repeatability and Reproducibilty"]
["RDT&E","Research, Development, Test & Evaluation"]
["REACH","Registration, Evaluation, Authorisation, and Restriction of Chemicals (Europe)"]
["RFP","Request for Proposal"]
["ROM","Rough Order of Magnitude"]
["S&T","Science & Technology"]
["SAF/AQ","Secretary of the Air Force for Acquisition"]
["SAP","Systeme, Anwendungen, Produkte (German for: Systems, Applications & Products)"]
["SASC","Senate Armed Services Committee"]
["SCM","Supply Chain Management"]
["SE","Systems Engineering or Support Equipment"]
["SEMP","System Engineering Master Plan"]
["SEMS","Systems Engineering Master Schedule"]
["SEP","Systems Engineering Plan"]
["SETR","Systems Engineering Technical Review (Process)"]
["SFR","System Functional Review"]
["SIE","Special Inspection Equipment"]
["SME","Subject Matter Expert"]
["SOO","Statement of Objectives"]
["SOW","Statement of Work"]
["SPC","Statistical Process Control"]
["SPM","System Program Manager"]
["SRR","System Requirements Review"]
["ST","Special Tooling"]
["STE","Special Test Equipment"]
["SVR","System Verification Review"]
["TBD","To Be Determined"]
["TDP","Technical Data Package"]
["TIM","Technical Interchange Meeting"]
["TMRR","Technology Maturation and Risk Reduction"]
["TRA","Technology Readiness Assessment"]
["TRL","Technology Readiness Level"]
["TRR","Test Readiness Review"]
["TTP","Technology Transition Plan"]
["USD(AT&L)","Under Secretary of Defense for Acquisition, Technology & Logistics"]
["VM","Virtual Manufacturing"]
["VR","Variability Reduction"]
["VSM","Value Stream Mapping"]
["WIP","Work in Process/Progress"]

{"sheet":"MRL Printable Matrix_V2020","index":8,"n_rows":27,"n_cols":12,"rows_kept":27,"cut":false,"hidden":false,"merged":[[23,0,24,0],[25,0,26,0],[4,0,6,0],[7,0,8,0],[9,0,11,0],[12,0,15,0],[16,0,18,0],[19,0,21,0],[0,1,0,11],[1,0,1,1],[1,2,1,4],[1,6,1,7],[1,8,1,9],[2,0,2,1]]}
["Version 2020","DoD Manufacturing Readiness Levels (MRLs)"]
["Acquisition Phase",null,"Pre-Materiel Development Decision (Pre-MDD)",null,null,"Materiel Solution Analysis (MSA) ","Technology Maturation and Risk Reduction (TMRR)",null,"Engineering & Manufacturing Development (EMD)",null,"Low-Rate Initial Production (LRIP)","Full-Rate Production (FRP)"]
["Technical Reviews",null,null,null,null,"ASR","SRR/SFR","PDR","CDR","PRR/SVR","PCA"]
["Thread","Sub-Thread","MRL 1","MRL 2","MRL 3","MRL 4","MRL 5","MRL 6","MRL 7","MRL 8","MRL 9","MRL 10"]
["A - Technology and Industrial Base","Technology\nMaturity ","Should be assessed at TRL 1.","Should be assessed at TRL 2.","Should be assessed at TRL 3.","Should be assessed at TRL 4.","Should be assessed at TRL 5.","Should be assessed at TRL 6.","Should be assessed at TRL 7","Should be assessed at TRL 7 or TRL 8. ","Should be assessed at TRL 8 or TRL 9.","Should be assessed at TRL 9."]
[null,"A.1\nIndustrial Base","Global trends in emerging industrial base capabilities identified.","Potential industrial base capability gaps identified.","Industrial base capabilities for potential sources identified for system concepts. ","Industrial base capabilities surveyed for preferred materiel solution, key technologies, components, and/or key processes. Industrial base capability risks and issues included in AoA.","Industrial base capabilities assessment initiated to identify potential manufacturing sources. Sole/single/ foreign source vendors and vendors of technologies with potential obsolescence issues identified and planning initiated to minimize risks. ","Industrial base capabilities assessment for MS B completed. Industrial capability in place to support manufacturing of development articles. Plans to minimize sole/single/foreign sources and obsolescence issues complete. Need for sole/single/foreign sources justified. Potential alternative sources identified. ","Industrial capability to support production analyzed. Sole/single/foreign sources, source stability, and obsolescence issues are assessed/monitored. Potential alternate sources developed if necessary. ","Industrial base capability assessment for MS C completed. Industrial capability is in place to support LRIP. Sources are available, including multi-sourcing where cost-effective or necessary to mitigate risk.","Industrial capability assessment for FRP has been completed and capability is in place to support start of FRP. ","Industrial capability supports FRP and is assessed to support modifications, upgrades, surge and other potential manufacturing requirements. "]
[null,"A.2\nManufacturing Technology Development","Global trends in manufacturing science and technology identified (i.e., concepts, capabilities).","Potential manufacturing science and technology gaps identified.","Manufacturing technology requirements identified to address potential capability gaps for system concepts.","Manufacturing technology development initiatives defined for preferred materiel solution. Manufacturing technology development requirements considered in the AoA.","Required manufacturing technology development efforts initiated.","Manufacturing technology efforts continuing. Required manufacturing technology development solutions demonstrated in a production relevant environment. ","Manufacturing technology efforts continuing. Required manufacturing technology development solutions demonstrated in a production representative environment. ","Primary manufacturing technology efforts concluding. Improvement efforts continuing. Required manufacturing technology solutions validated on a pilot line.","Manufacturing technology process improvements efforts initiated for FRP.","Manufacturing technology continuous process improvements ongoing. "]
["B - Design ","B.1\nProducibility Program","Hypotheses developed for cause-effect relationships between technology variables and producibility.","Studies performed to test hypotheses regarding cause-effect relationships between technology variables and producibility. Elements identified which have a potential impact to producibility (i.e., materials, processes, capabilities, limitations).","System concept elements evaluated for manufacturability and producibility using experiments, modeling, and simulation.","Initial producibility and manufacturability assessments in selection of preferred materiel solution completed. Results considered in AoA documented in AS key components/technologies. ","Producibility and manufacturability assessments of key technologies and components initiated. Ongoing design trades consider manufacturing processes and industrial base capability constraints. Manufacturing processes assessed for capability to be tested and verified in production. Manufacturing processes assessed for influence on O&S.","Producibility assessments and producibility trade studies (performance vs. producibility) of key technologies/components completed. Results used to shape AS, SEP, manufacturing and producibility plans, and planning for EMD or technology insertion programs. Preliminary design choices assessed against manufacturing processes and industrial base capability constraints. Producibility enhancement efforts (i.e., DFM, DFA, etc.) initiated.","Detailed producibility trade studies using knowledge of key design characteristics and related manufacturing process capability completed. Producibility enhancement efforts (i.e., DFM, DFA, etc.) ongoing for optimized integrated system. Manufacturing processes re-assessed as needed for capability to be tested and verified. Manufacturing processes re-assessed as needed for potential influence on O&S.","Producibility improvements implemented on system. Known producibility risks and issues managed for LRIP. ","Prior producibility improvements analyzed for effectiveness during LRIP. Producibility risks and issues discovered in LRIP managed for FRP. ","Design producibility improvements demonstrated in FRP. Process producibility improvements ongoing. All modifications, upgrades, DMSMS and other changes assessed for producibility. "]
[null,"B.2\nDesign Maturity","Current capability deficiencies and gaps identified.","Analyses performed to evaluate the feasibility of potential solutions to address capability gaps. ","High level performance, lifecycle, and technical requirements defined and evaluated for system concepts. Trade-offs in design options based on experiments and initial MOEs.","Form, fit, and function constraints, and manufacturing capabilities identified for preferred systems concept. SEP and T&E Strategy recognize the need for the establishment/validation of manufacturing capability and management of manufacturing risk for the product lifecycle. Initial KPPs identified for preferred systems concept. System characteristics and measures to support required capabilities identified.","Lower level performance requirements sufficient to proceed to preliminary design. All enabling/critical technologies and components identified and the product lifecycle considered. Evaluation of the design for KCs initiated. Product data required for prototype component manufacturing released.","System allocated baseline established. Product requirements and features are well enough defined to support PDR. Product data essential for subsystem/ system prototyping has been released, and all enabling/critical components have been prototyped. Preliminary KCs for the design identified and mitigation plans initiated.","Product design and features are well enough defined to support CDR, even though design change traffic may be significant. All product data essential for component manufacturing released. Potential KC risks and issues identified with mitigation plans in place.","Detailed design of product features and interfaces completed. All product data essential for system manufacturing released. Design change traffic does not significantly impact LRIP. KCs are attainable based upon pilot line demonstrations.","Major product design features and configuration are stable. System design has been validated through operational testing of LRIP items. PCA or equivalent complete as necessary. Design change traffic is limited. All KCs are controlled in LRIP to appropriate quality levels.","Product design is stable. Design changes are few and generally limited to those required for continuous improvement or in reaction to obsolescence. All KCs are controlled in FRP to appropriate quality levels."]
["C - Cost & Funding","C.1\nProduction Cost Knowledge\n(Cost modeling)","Hypotheses developed regarding technology impact on affordability.","Cost model approach defined.","Manufacturing cost estimates for system concepts developed. Initial cost models developed which include high-level process steps and materials.","Cost estimates refined based on anticipated production volumes associated with preferred materiel solution. Cost model updated with identified cost drivers (i.e., process variables, manufacturing, materials, and special requirements). Cost driver uncertainty quantified. Cost model supports AoA and ASR. ","Prototype components produced in a production relevant environment, or simulations drive end-to-end cost models. Cost model includes materials, labor, equipment, tooling/STE/SIE, setup, yield/scrap/rework, WIP, and capability/capacity constraints. ","Cost model updated with design requirements, material specifications, tolerances, IMS, results of system/subsystem simulations and production relevant prototype demonstrations. ","Cost model updated with the results of systems/sub-systems produced in a production representative environment, production plant layout and design, and obsolescence solutions.","Cost model updated with results of pilot line build.","FRP cost model updated with result of LRIP build. ","Cost model validated against actual FRP cost. "]
[null,"C.2\nCost Analysis","Initial manufacturing and quality costs identified.","Potential manufacturing and quality cost drivers and system affordability gaps identified. ","Analyses conducted to refine manufacturing and quality cost drivers, risks, and development strategy (e.g., lab to pilot to factory). Potential cost reduction and system affordability gap closure strategies identified.","Producibility and lifecycle cost risks and issues assessed for preferred materiel solution. Initial cost analysis supports AoA and ASR.","Costs analyzed using prototype component actuals to ensure target costs are achievable. Decisions regarding design choices, make/buy, capacity, process capability, sources, quality, KCs, yield/rate, and variability influenced by cost models. ","Costs analyzed using prototype system/sub-system actuals to ensure target costs are achievable. Cost targets allocated to subsystems. Cost reduction and avoidance strategies developed. Manufacturing cost drivers for \"Should-Cost\" model provided.","Manufacturing costs rolled up to system/sub-system level and tracked against targets. Detailed trade studies and engineering change requests supported by cost estimates. Cost reduction and avoidance strategies underway. Manufacturing cost drivers for \"Should-Cost\" model updated.","Costs analyzed using pilot line actuals to ensure target costs are achievable. Manufacturing cost analysis supports proposed changes to requirements or configuration. Cost reduction initiatives ongoing. Manufacturing cost drivers for \"Should-Cost\" model updated.","LRIP cost goals met and learning curves analyzed with actual data. Cost reduction initiatives ongoing. Touch labor efficiency analyzed to meet production rates and elements of inefficiency are identified with plans in place for reduction.","FRP cost goals met. Cost reduction initiatives ongoing."]
[null,"C.3\nManufacturing Investment Budget","Potential manufacturing investment strategy developed.","Program/projects have reasonable budget estimates for reaching MRL 3 through experiment. Manufacturing investment budget ROM estimates identified to support industrial base and manufacturing capability gap closure strategies.","Program/projects have reasonable budget estimates for reaching MRL 4 by MS A. Preliminary manufacturing investment budget estimates for manufacturing gap closure recommendations have been developed.","Manufacturing technology budget initiatives developed and incorporated to reduce costs. Program has reasonable budget estimate for reaching MRL 6 by MS B. Estimate includes capital investment for production relevant equipment. All outstanding MRL 4 risks and issues understood with approved mitigation plans in place. ","Program has updated budget estimate for reaching MRL 6 by MS B. All outstanding MRL 5 risks and issues understood with approved mitigation plans in place. ","Program has reasonable budget estimate for reaching MRL 8 by MS C. Estimate includes capital investment for production-representative equipment by CDR and pilot line equipment by MS C. All outstanding MRL 6 risks and issues understood with approved mitigation plans in place. ","Program has updated budget estimate for reaching MRL 8 by MS C. All outstanding MRL 7 risks and issues understood with approved mitigation plans in place. ","Program has reasonable budget estimate for reaching MRL 9 by the FRP decision point. Estimate includes investment for LRIP and FRP. All outstanding MRL 8 risks and issues understood with approved mitigation plans in place. ","Program has reasonable budget estimate for FRP. All outstanding MRL 9 risks and issues understood with approved mitigation plans in place. ","Production budgets sufficient for production at required rates and schedule to support funded program. "]
["D - Materials (Raw Materials, Components, Sub-assemblies and Sub-systems)","D.1\nMaturity","New material properties and characteristics surveyed and identified for research (e.g., manufacturability, quality).","Potential effects of new material properties on design application manufacturability and quality predicted based on research.","Effects of new material properties on design concept manufacturability and quality validated using experiments and models. ","New materials and components for preferred materiel solution demonstrated in a laboratory environment.","Materials manufactured or produced in a prototype environment (may be in a similar application/program). Maturation efforts in place to address new material production risks for technology demonstration.","Material maturity verified through technology demonstration articles. Preliminary material specifications in place. Material properties adequately characterized.","Material maturity sufficient for pilot line build. Material specifications approved.","Materials proven and validated during EMD as adequate to support LRIP. Material specifications stable. ","Materials controlled to specifications in LRIP. Materials proven and validated as adequate to support FRP.","Materials controlled to specifications in FRP. "]
[null,"D.2\nAvailability","Global trends for material availability, obsolescence, and DMSMS surveyed and identified for research.","Material availability, obsolescence, and DMSMS gaps identified.","Material availability, obsolescence, and DMSMS gap closure strategy defined.","Projected lead times identified for all difficult to obtain, difficult to process, or hazardous materials. Quantities and lead times estimated. Material availability risks and issues for preferred materiel solution considered in AoA. Mitigation plans incorporated in SEP for the preferred system concept.","Availability risks and issues addressed for prototype build. Significant material risks identified for all materials. Planning initiated to address scale-up issues. ","Availability risks and issues addressed to meet EMD build. Long-lead items identified. Potential obsolescence issues identified. Components assessed for future DMSMS risk. ","Availability risks and issues addressed to meet LRIP builds. Long lead procurements identified and mitigated. Obsolescence plan in place. DMSMS mitigation strategies for components in place.","Availability risks and issues managed for LRIP. Long lead procurement initiated for LRIP. Availability issues addressed to meet FRP builds.","Long lead procurement initiated for FRP. Availability risks and issues managed for FRP. ","All material availability risks and issues managed. "]
[null,"D.3\nSupply Chain Management","Global trends for supply chain capability and capacity surveyed. ","Potential supply chain capability and capacity gaps identified.","Supply chain capability and capacity gap closure strategies defined.","Survey of potential supply chain sources for preferred materiel solution completed. Supply chain capability and capacity analyses considered in the AoA.","Potential supply chain sources identified and evaluated as able to support prototype build.","Lifecycle Supply Chain requirements updated. Critical suppliers list updated. Supply chain plans in place (e.g. teaming agreements, etc.) supporting an EMD contract award.","Effective supply chain management processes defined, documented, and in place. Plan developed for predictive indicators. Assessment of critical first tier supply chain completed (i.e., capability, capacity, etc.). ","Assessment of critical second and lower tier supply chain completed. Robust requirements flow down processes in place and verified. Supplier compliance with program requirements and changes validated. Plan for predictive indicators for use in production updated. Supply chain adequate to support LRIP.","Long term agreements in place where practical. Prime's supplier management metrics (including thresholds and goals) in place and used to manage risks. Predictive indicators to manage suppliers in place. Supply chain is stable and adequate to support FRP.","Supply chain proven and supports FRP requirements."]
[null,"D.4\nSpecial Handling \n(i.e. GFP, shelf life, security, hazardous materials, storage environment, ESH, etc.) ","Hazardous materials identified and safety procedures in place. ","Initial evaluation of potential regulatory requirements and special handling concerns completed.\nRaw materials and components assessed for special handling and potential regulatory requirements.","ESH compliance risk identified. List of hazardous materials identified and alternatives evaluated. Special handling procedures applied in the lab. Special handling concerns assessed. ","ESH compliance risk mitigated in lab environment. List of hazardous materials updated and alternatives assessed. Special handling procedures applied and disposal procedures evaluated. Special handling requirements identified, analyzed, and documented in the SEP. ","ESH requirements and special handling procedures applied in production relevant environment. Special handling requirement gaps identified. New special handling processes demonstrated in lab environment. ","ESH requirements addressed and documented. Special handling procedures demonstrated in production relevant environment. Plans to address special handling requirement gaps, risks, and issues complete. Manufacturing assessed for material storage and waste handling risks.","ESH compliance demonstrated in production representative environment. Special handling procedures applied in production representative environment. Special handling procedures developed and annotated on work instructions for pilot line. Hazardous material storage and disposal plan in place for the pilot line.","ESH compliance demonstrated in pilot line production. Special handling procedures applied in pilot line environment and demonstrated in EMD or technology insertion programs. Special handling risks and issues managed for LRIP. All work instructions contain special handling provisions as required. Hazardous material storage and disposal plan evaluated and in place for LRIP.","ESH compliance demonstrated in LRIP. Special handling, and hazardous material storage and disposal procedures demonstrated in LRIP environment. Special handling, and hazardous material storage and disposal risks and issues managed for FRP. ","ESH compliance demonstrated in FRP. Special handling and hazardous material storage and disposal procedures effectively implemented in FRP. "]
["E - Process Capability & Control","E.1\nModeling & Simulation (Product & Process)","Modeling and simulation approaches/tools identified to support manufacturing and quality activities.","Modeling and simulation in development initiated.","Manufacturing and quality gaps for system concepts identified using modeling and simulation.","Production modeling and simulation tools utilized to define manufacturing and quality requirements for preferred materiel solution. Modeling and simulation results considered in the AoA.","Initial modeling & simulations (product or process) developed at the component level and used to determine constraints.","Initial modeling & simulations developed at the sub-system or system level, and used to determine system constraints.","Modeling & simulations used to determine system constraints and to identify improvement opportunities. ","Modeling & simulations verified by pilot line build. Results used to improve process and demonstrate that LRIP requirements can be met. ","Modeling & simulations verified by LRIP build, assist in management of LRIP, and demonstrate that FRP requirements can be met. ","Modeling & simulations verified by FRP build. Production simulation models used as tools to assist in management of FRP. "]
[null,"E.2\nManufacturing Process Maturity","Hypotheses developed regarding cause-effect relationships between process variables and process stability and repeatability.","Studies performed to test hypotheses regarding cause-effect relationships. Initial process approaches identified.","Cause-effect relationships between process control variables and process stability and repeatability validated through laboratory experiments. Critical process control variables identified. ","Maturity of critical processes for preferred materiel solution assessed.\nProcess capability requirements and improvement plans developed and documented in the SEP. ","Process Maturity assessed on similar processes in production. Process capability requirements identified for pilot line, LRIP and FRP.","Manufacturing processes demonstrated in production relevant environment. Collection or estimation of process capability data from prototype build and refinement of process capability requirements initiated.","Manufacturing processes demonstrated in a production representative environment. Collection and/or estimation of process capability data and refinement of process capability requirements ongoing.","Manufacturing processes for LRIP verified on a pilot line. Process Capability data from pilot line meets target. Process capability requirements for LRIP and FRP refined based upon pilot line data.","Manufacturing processes are stable, adequately controlled, capable, and have achieved program LRIP objectives. Variability experiments conducted to show FRP impact and potential for continuous improvement. ","Manufacturing processes are stable, adequately controlled, capable, and have achieved program FRP objectives. "]
[null,"E.3\nProcess Yields and Rates","Hypotheses developed regarding future state manufacturing yields and rates.","Studies performed to test hypotheses regarding yields and rates.","Initial estimates of yields and rates for system concepts identified through laboratory experiments. Yield and rate gaps for system concepts identified.","Yield and rate assessments on preferred materiel solution completed and considered in the AoA. Yield and rate gap closure strategies identified for the preferred materiel solution and documented in the SEP.","Target yields and rates established for pilot line, LRIP, and FRP. Yield and rate issues identified. Improvement plans developed/initiated. ","Yields and rates from production relevant environment evaluated against targets and the results feed improvement plan.","Yields and rates from production representative environment evaluated against pilot line targets and the results feed improvement plans.","Pilot line targets achieved. Yields and rates required to begin LRIP refined using pilot line results. Improvement plans ongoing and updated.","LRIP yield and rate targets achieved. Yields and rates required to begin FRP refined using LRIP results. Yield improvements on-going. ","FRP yield and rate targets achieved. Yield improvements on-going. "]
["F - Quality","F.1\nQuality Management ","Quality management considerations surveyed and included in early planning activities","Quality management needs assessed, analyzed, and validated.","Quality management requirements for system concepts identified.","Quality strategy for the preferred materiel solution developed, considered in the AoA, and documented in the SEP and the AS. ","Quality strategy updated to reflect KC identification activities. ","Initial Quality Plan and QMS are in place. Quality risks, issues, and metrics have been identified and improvement plans initiated. ","Quality targets established. QMS elements (i.e., control of nonconforming material, corrective action, etc.) meet requirements of appropriate industry standards. Program-specific Quality Program Plan developed. ","Program-specific Quality Program Plan established. Program Quality Manager assigned. Quality targets assessed against pilot line, results feed continuous quality improvements. ","Quality targets verified on LRIP line. Continuous quality improvement on-going. Management review of Quality measures conducted on regular basis and appropriate actions taken.","Quality targets verified on FRP line. Continuous quality improvement on-going. Statistical controls applied where appropriate. "]
[null,"F.2\nProduct Quality","Quality metrology state of the art surveyed. Hypotheses developed regarding cause-effect relationships between technology variables and quality.","Studies performed to test hypotheses regarding cause-effect relationships between technology variables and quality. Elements identified which have a potential impact on quality (i.e., materials, processes, capabilities, limitations). ","System concept elements evaluated for quality using experiments, modeling and simulation. Initial product quality requirements, risks, and issues identified. Inspection technologies identified.","Product quality requirements and the inspection and acceptance testing strategy for the preferred materiel solution considered in AoA and documented in the AS. Product quality risk and issue mitigation plans documented in the SEP.","Roles and responsibilities identified for acceptance test procedures, in-process and final inspections, and statistical process controls for prototype units. ","KC management approach defined. Initial requirements identified for acceptance test procedures and in-process and final inspection requirements for EMD units. Appropriate inspection and acceptance test procedures identified for prototype units. ","Quality data from the production representative environment collected and analyzed and results used to shape improvement plans. Control plans completed for management of KCs. Test and Inspection plans being developed for EMD units. ","KCs managed. Measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.). Pilot line data meets capability requirements for all KCs. Test and Inspection plans complete and validated for production units.","Data from LRIP demonstrates production processes, for all KCs and other manufacturing processes critical to quality, are capable and under control for FRP.","KCs controlled at rate. Results achieve targeted statistical level on all KCs. Results reflect continuous improvement."]
[null,"F.3\nSupplier Quality/ Management","Supplier quality and quality management systems state of the art surveyed.","Initial supplier quality and quality management systems evaluated.","Supplier quality and quality management system requirements for system concepts identified.","Potential supplier quality capabilities, risks, and issues identified for the preferred materiel solution, including subtier suppliers. Supplier quality management system requirements defined, and documented in the AS. ","Supply base quality capabilities and risks identified, including subtier supplier quality management.","Supply base quality improvement initiatives identified addressing supplier QMS shortfalls, including subtier supplier quality management. ","Key supplier QMSs meet appropriate industry standards. Supplier quality data from production representative units collected and analyzed. Strategy for audits of critical supplier processes outlined. ","Supplier program-specific QMSs adequate. Supplier products qualification testing and first article inspection completed. Acceptance testing of supplier products adequate to begin LRIP. Plan for subcontractor process audits in place and implemented by prime contractor.","Supplier quality management of KCs and other critical manufacturing processes demonstrates capability and control for FRP. Acceptance testing of supplier products reflects control of quality adequate to begin FRP. Subcontractor quality audits performed as necessary to ensure subcontractor specification compliance.","Supplier quality data reflects adequate management of KCs and control of critical manufacturing processes, including quality management down to subtier suppliers. Results achieve high statistical level (e.g., 6-sigma) on all critical dimensions. Subcontractor quality audits performed as necessary to ensure subcontractor specification compliance."]
["G - Manufacturing Workforce (Engineering & Production)","G.1\nManufacturing Workforce (Engineering & Production)","Workforce skill sets to support emerging trends in manufacturing and technology surveyed.","Workforce skill sets to support emerging trends in manufacturing and technology evaluated.","Workforce skill set requirements for system concepts identified. Workforce skill set capability gaps identified","Workforce skill set and production workforce requirements (technical and operational) for the preferred materiel solution identified and considered in the AoA. Workforce training and development requirements to close skill set gaps defined. Availability of workforce for TMRR Phase determined.","Skill sets identified and plans developed to meet prototype and production needs. Special skills certification and training requirements established. ","Manufacturing workforce skills available for the production relevant environment. Resources (quantities and skill sets) identified and initial plans developed to achieve requirements for pilot line and production. ","Manufacturing workforce resource requirements identified and plans developed to achieve pilot line requirements. Plans to achieve LRIP workforce requirements updated. Pilot line workforce trained in production representative environment.","Manufacturing workforce resource requirements identified and plans developed to achieve LRIP requirements. LRIP personnel trained on pilot line where possible. Plans to achieve FRP workforce requirements initiated based on pilot line. \n","LRIP personnel requirements met. Plan to achieve FRP workforce requirements implemented. ","FRP personnel requirements met. Production workforce skill sets maintained in spite of workforce attrition. "]
["H - Facilities","H.1\nTooling/STE/SIE","State of the art tooling, test and inspection equipment surveyed.","Potential tooling, STE, and SIE requirements identified.","Tooling, STE, and SIE requirements for system concepts identified. Special tooling, STE, and SIE gaps identified.","Tooling/STE/SIE requirements for the preferred materiel solution considered as part of AoA.","Tooling and STE/SIE requirements identified with supporting rationale and schedule.\n","Prototype tooling and STE/SIE concepts demonstrated in production relevant environment. Requirements development efforts for production tooling and STE/SIE complete.","Design and development efforts for production tooling and STE/SIE initiated with STE/SIE validation plans complete. Manufacturing equipment maintenance strategy developed.","Tooling, test and inspection equipment proven on pilot line and additional requirements identified for LRIP. STE/SIE validated as part of pilot line validation IAW validation plan. Manufacturing equipment maintenance demonstrated on pilot line.","All tooling, test and inspection equipment proven in LRIP and additional requirements identified for FRP. Manufacturing equipment maintenance schedule demonstrated. STE/SIE validation maintained as necessary.","Proven tooling, test and inspection equipment in place to support maximum FRP. Planned equipment maintenance schedule achieved. STE/SIE validation maintained as necessary."]
[null,"H.2 \nFacilities","Current facility capabilities and capacity surveyed.","Potential facility capabilities and capacity requirements identified.","Facility capabilities and capacity requirements and gaps for system concepts identified. ","Capability and availability of manufacturing facilities for prototype development and production of the preferred materiel solution evaluated, included in the AoA, and documented in the AS and SEP. Human factors & ergonomics and safety requirements for manufacturing (personnel, processes & equipment) identified.","Manufacturing facilities identified and plans developed to produce prototypes. Human factors & ergonomics and safety requirements for manufacturing (personnel, processes & equipment) assessed.","Manufacturing facilities identified and plans developed to produce pilot line build. Human factors & ergonomics and safety requirements for manufacturing (personnel, processes & equipment) verified in a production relevant environment.","Manufacturing facilities identified and plans developed to produce LRIP build. Human factors & ergonomics and safety practices for manufacturing (personnel, processes & equipment) validated in a production representative environment. ","Pilot line facilities demonstrated. Manufacturing facilities adequate to begin LRIP. Plans in place to support transition to FRP. Workplace safety is adequate. Human factors & ergonomics and safety practices for manufacturing (personnel, processes & equipment) demonstrated on a pilot line.","Manufacturing facilities in place and demonstrated in LRIP. Capacity plans adequate to support FRP. Human factors & ergonomics and safety practices for manufacturing (personnel, processes & equipment) demonstrated in LRIP .","Production facilities in place and capacity demonstrated to meet maximum FRP requirements. Human factors & ergonomics and safety requirements for manufacturing (personnel, processes & equipment) demonstrated in FRP."]
["I - Manufacturing Management","I.1\nManufacturing Planning & Scheduling","Manufacturing management considerations surveyed and included in early planning activities.","Manufacturing management needs assessed, analyzed and validated.","Manufacturing management requirements for system concepts identified. ","Manufacturing strategy for the preferred materiel solution developed, considered in the AoA, and documented in the AS. Prototype schedule risk mitigation efforts documented in the SEP. ","Manufacturing strategy refined based upon preferred concept. Prototype schedule risk mitigation efforts initiated. ","Initial manufacturing approach developed. All system design related manufacturing events included in IMP/IMS. Manufacturing risk, and issue mitigation approach for pilot line and/or technology insertion programs defined. ","Initial Manufacturing Plan developed and included in IMP/IMS. Manufacturing risks and issues integrated into mitigation plans. Initial work instructions developed. Effective production control system in place to support pilot line. ","Manufacturing Plan updated for LRIP. All manufacturing risks and issues identified and assessed with approved mitigation plans in place. Work instructions finalized. Effective production control system in place to support LRIP.","Manufacturing plan updated for FRP. All manufacturing risks and issues managed. Effective production control system in place to support FRP.","All manufacturing risks and issues managed."]
[null,"I.2\nMaterials Planning ","Materials planning state of the art surveyed.","Initial availability, lead time, handling and storage requirements for potential materials and components evaluated.","Materials and components list for system concepts developed. Initial materials planning requirements (i.e., availability, lead times, handling, and storage) identified.","Materials and components list with estimates for availability, lead times, handling and storage requirements developed and considered in the AoA. ","Make/buy evaluations initiated and include production considerations for pilot line, LRIP, and FRP needs. Lead times and other materials risks and issues identified. ","Most material make/buy decisions complete, material risks and issues identified, and mitigation plans developed. BOM initiated. ","Make/Buy decisions and BOM complete for pilot line build. Material planning systems in place for pilot line build. ","Make/Buy decisions and BOM complete to support LRIP. Material planning systems proven on pilot line for LRIP build.","Make/Buy decisions and BOM complete to support FRP. Material planning systems proven in LRIP and sufficient for FRP.","Material planning systems validated on FRP build. "]

{"sheet":"Questionnaire","index":9,"n_rows":562,"n_cols":11,"rows_kept":562,"cut":false,"hidden":false,"merged":[]}
[]
[]
["MRL FILTER","MRL","Thread","Sub","Question","Purpose","Additional Considerations","Lessons Learned","Answer","Evidence","Meets Criteria"]
[1,1,"A - Technology & Industrial Base","A.0 - Technology Maturity","Is the Technology Readiness at TRL 1 or greater?","","",""]
[2,2,"A - Technology & Industrial Base","A.0 - Technology Maturity","Is the Technology Readiness at TRL 2 or greater?","","",""]
[3,3,"A - Technology & Industrial Base","A.0 - Technology Maturity","Is the Technology Readiness at TRL 3 or greater?","","",""]
[4,4,"A - Technology & Industrial Base","A.0 - Technology Maturity","Is the Technology Readiness at TRL 4 or greater?","","",""]
[5,5,"A - Technology & Industrial Base","A.0 - Technology Maturity","Is the Technology Readiness at TRL 5 or greater?","","",""]
[6,6,"A - Technology & Industrial Base","A.0 - Technology Maturity","Is the Technology Readiness at TRL 6 or greater?","","",""]
[7,7,"A - Technology & Industrial Base","A.0 - Technology Maturity","Is the Technology Readiness at TRL 7 or greater?","","",""]
[8,8,"A - Technology & Industrial Base","A.0 - Technology Maturity","Is the Technology Readiness at TRL 7 or 8?","","",""]
[9,9,"A - Technology & Industrial Base","A.0 - Technology Maturity","Is the Technology Readiness at TRL 8 or 9?","","",""]
[10,10,"A - Technology & Industrial Base","A.0 - Technology Maturity","Is the Technology Readiness at TRL 9?","","",""]
[11,1,"A - Technology & Industrial Base","A.1 - Industrial Base","Have global trends in emerging industrial base capabilities been identified?","Purpose: \nTo identify possible sources for a variety of materials that may be used in the system and understand the state of the art with respect to processes. Understanding the materials and processes (M&P) state of the art provides a foundation of essential information for an Analysis of Alternatives with respect to system design and capabilities.","Additional Considerations: \n• Do you possess the most recent information regarding industrial base capabilities?\n• Are any strategic or critical materials obtained only from sole or foreign sources?\n• Are sources for critical materials financially sound with multiple products and limited dependence on military products for profitability?\n• Do critical materials have significant commercial application or have application to more than one service?\n• Are critical materials on the list for diminishing manufacturing sources/material shortages?\n• What are the limitations on using certain materials for US defense purposes?","Lessons Learned: \n• Sources previously available for certain materials and/or processes may no longer exist; therefore, it is essential that the industrial base analysis be up-to-date.\n• For certain systems/programs, the use of foreign sources may either be proscribed or limited due to the strategic nature of the materials. \n• Foreign sources for some materials or processes may refuse to sell to US weapons programs, in which case, workarounds must be established early in the system life-cycle."]
[12,2,"A - Technology & Industrial Base","A.1 - Industrial Base","Have potential industrial base capability gaps been identified?","Purpose: \nTo narrow down possible sources for materials and processes applying to Critical Technologies (CTs) for preferred system design alternatives. Understanding the materials and processes (M&P) availability of sources provides essential information on key processes and technology gaps that may exist when one does an Analysis of Alternatives in the Technology Development (TD) phase. The Alternative Systems Review also answers the question, “can this solution be developed in a timely manner at an acceptable level of risk?” Manufacturing feasibility is of critical importance in this review to increase the probability of meeting the cost and schedule constraints for each alternative concept.","Additional Considerations:\n• Are industrial base studies current and do you possess the most recent information regarding industrial capabilities that apply to CTs?\n• Are any strategic/critical materials needed for CTs obtained only from sole/foreign sources?\n• Are sources for critical materials financially sound with multiple products and limited dependence on military products for profitability?\n• Do critical materials have significant commercial application or have application to more than one service?\n• Are critical materials on the list for diminishing manufacturing sources/material shortages?\n• What are the limitations on using certain materials for US defense purposes?\n• Have the preliminary manufacturing processes and risks been identified for prototypes?\n• Have required investments for technology development, to mature design and manufacturing related technologies, been identified and funded?\n• Have initial producibility assessments of design concepts been completed?","Lessons Learned: \n• Sources previously available for certain materials and/or processes may no longer exist; therefore, it is essential that the industrial base analysis be up-to-date.\n• For certain systems/programs, the use of foreign sources may either be proscribed or limited due to the strategic nature of the materials. \n• Foreign sources for some materials or processes may refuse to sell to US weapons programs, in which case, workarounds must be established early in the system life-cycle.\n• As designs fluctuate, Industrial Capability Analyses need to be constantly updated.\n• Fragility of the IB and criticality of critical components needs to be considered in the IBCA."]
[13,3,"A - Technology & Industrial Base","A.1 - Industrial Base","Have industrial base capabilities for potential sources been identified for system concepts? "]
[14,4,"A - Technology & Industrial Base","A.1 - Industrial Base","Have industrial base capabilities been surveyed for preferred materiel solution, key technologies, components, and/or key processes? ","Purpose: \nTo perform a thorough assessment of sources for materials and processes (M&P) applying to Critical Technologies (CTs) of the major system design alternatives. Understanding capability of the defense industrial base to support production of various alternative designs is crucial information for the AoA and the acquisition decision makers. For applicable products, the Acquisition Strategy should also address the approach to making production rate and quantity changes (i.e., surge) in response to contingency needs.\n","Additional Considerations: \n• Are industrial base studies current and do you possess the most recent information regarding industrial capabilities that apply to CTs?\n• Are any strategic or critical materials needed for CTs obtained only from sole/foreign sources?\n• Is there likely to be a future requirement to surge production of this system?\n• What is the viability of any associated essential industrial/technological capabilities?\n• What is the potential viability of non-selected firms as enduring competitors for defense products?\n• What span of time between current and potential future contract awards should be considered that will make selection critical to supplier business decisions?\n• Are there other businesses of the same type or emerging capabilities that could serve as a replacement solution?\n• What decisions will impact a supplier's future viability (jeopardize future competitiveness or do not provide a sufficient business case to keep the capabilities/unit around for the future)?\n• What decisions will establish new industrial capabilities (develop new facilities, demonstrate and \"productionize\" new technologies, preserve health of the industrial base)?\n• What priority under the Defense Priorities and allocation system (DPAS) is this program likely to have (e.g., DO, DX, etc.)?\n","Lessons Learned: \n• Sources previously available for certain materials and/or processes may no longer exist; therefore, it is essential that the industrial base analysis be up-to-date.\n• For certain systems/programs, the use of foreign sources may either be proscribed or limited due to the strategic nature of the materials. \n• Foreign sources for some materials or processes may refuse to sell to US weapons programs, in which case, workarounds must be established early in the system life-cycle.\n• As designs fluctuate, Industrial Base Capability Analyses need to be constantly updated.\n• Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n• Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished IBCA.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA.\n"]
[15,4,"A - Technology & Industrial Base","A.1 - Industrial Base","Are industrial base capability risks and issues included in AoA?","Purpose: \nTo perform a thorough assessment of sources for materials and processes (M&P) applying to Critical Technologies (CTs) of the major system design alternatives. Understanding capability of the defense industrial base to support production of various alternative designs is crucial information for the AoA and the acquisition decision makers. For applicable products, the Acquisition Strategy should also address the approach to making production rate and quantity changes (i.e., surge) in response to contingency needs.\n","Additional Considerations: \n• Are industrial base studies current and do you possess the most recent information regarding industrial capabilities that apply to CTs?\n• Are any strategic or critical materials needed for CTs obtained only from sole/foreign sources?\n• Is there likely to be a future requirement to surge production of this system?\n• What is the viability of any associated essential industrial/technological capabilities?\n• What is the potential viability of non-selected firms as enduring competitors for defense products?\n• What span of time between current and potential future contract awards should be considered that will make selection critical to supplier business decisions?\n• Are there other businesses of the same type or emerging capabilities that could serve as a replacement solution?\n• What decisions will impact a supplier's future viability (jeopardize future competitiveness or do not provide a sufficient business case to keep the capabilities/unit around for the future)?\n• What decisions will establish new industrial capabilities (develop new facilities, demonstrate and \"productionize\" new technologies, preserve health of the industrial base)?\n• What priority under the Defense Priorities and allocation system (DPAS) is this program likely to have (e.g., DO, DX, etc.)?\n","Lessons Learned: \n• Sources previously available for certain materials and/or processes may no longer exist; therefore, it is essential that the industrial base analysis be up-to-date.\n• For certain systems/programs, the use of foreign sources may either be proscribed or limited due to the strategic nature of the materials. \n• Foreign sources for some materials or processes may refuse to sell to US weapons programs, in which case, workarounds must be established early in the system life-cycle.\n• As designs fluctuate, Industrial Base Capability Analyses need to be constantly updated.\n• Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n• Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished IBCA.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA.\n"]
[16,5,"A - Technology & Industrial Base","A.1 - Industrial Base","Has an assessment of industrial base capabilities been initiated to identify potential manufacturing sources?","Purpose: \nTo perform a thorough assessment of sources for materials and processes (M&P) applying to Critical Technologies (CTs) of the major system design alternatives. Understanding capability of the defense industrial base to support production of various alternative designs is crucial information for the AoA and the acquisition decision makers. For applicable products, the Acquisition Strategy should also address the approach to making production rate and quantity changes (i.e., surge) in response to contingency needs.\n","Additional Considerations: \n• Are industrial base studies current and do you possess the most recent information regarding industrial capabilities that apply to CTs?\n• Are any strategic or critical materials needed for CTs obtained only from sole/foreign sources?\n• Is there likely to be a future requirement to surge production of this system?\n• What is the viability of any associated essential industrial/technological capabilities?\n• What is the potential viability of non-selected firms as enduring competitors for defense products?\n• What span of time between current and potential future contract awards should be considered that will make selection critical to supplier business decisions?\n• Are there other businesses of the same type or emerging capabilities that could serve as a replacement solution?\n• What decisions will impact a supplier's future viability (jeopardize future competitiveness or do not provide a sufficient business case to keep the capabilities/unit around for the future)?\n• What decisions will establish new industrial capabilities (develop new facilities, demonstrate and \"productionize\" new technologies, preserve health of the industrial base)?\n• What priority under the Defense Priorities and allocation system (DPAS) is this program likely to have (e.g., DO, DX, etc.)?\n","Lessons Learned: \n• Sources previously available for certain materials and/or processes may no longer exist; therefore, it is essential that the industrial base analysis be up-to-date.\n• For certain systems/programs, the use of foreign sources may either be proscribed or limited due to the strategic nature of the materials. \n• Foreign sources for some materials or processes may refuse to sell to US weapons programs, in which case, workarounds must be established early in the system life-cycle.\n• As designs fluctuate, Industrial Base Capability Analyses need to be constantly updated.\n• Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n• Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished IBCA.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA.\n"]
[17,5,"A - Technology & Industrial Base","A.1 - Industrial Base","Have sole/single/ foreign source vendors and vendors of technologies with potential obsolescence issues been identified? "]
[18,5,"A - Technology & Industrial Base","A.1 - Industrial Base","Does sole/single/ foreign source vendors and vendors of technologies with potential obsolescence issues have planning initiated to minimize risks?"]
[19,6,"A - Technology & Industrial Base","A.1 - Industrial Base","Is the assessment of industrial base capabilities to support the Milestone (MS) B decision complete?","Purpose: \nAssure a thorough assessment of sources for materials and processes that apply to Critical Technologies (CTs) has been completed and that viable sources and potential alternatives have been identified. Assuring that the capability of the defense industrial base to support production of system and subsystem designs is crucial for Milestone B decision makers and should be an exit criterion for PDR.\n","Additional Considerations:\n• Have the majority of manufacturing processes been defined and characterized?\n• Are initial manufacturing approaches documented?\n• Have producibility assessments of key technologies been completed?\n• Has a production cost model been constructed?\n• Can the industrial base support production of development articles?\n• Have long-lead and key supply chain elements been identified? \n• What decisions will impact a supplier's future viability (jeopardize future competitiveness or do not provide a sufficient business case to keep the capabilities/unit around for the future)?\n• What is the certainty of future demand for key products given an economic turndown?\n• What decisions will establish new industrial capabilities (develop new facilities, demonstrate and \"productionize\" new technologies, preserve health of the industrial base)?","Lessons Learned: \n• As designs change, Industrial Base Capability Analyses need to be constantly updated.\n• Failure to consider Industrial base capability as part of the PDR may lead to jeopardizing future production schedules.\n• Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n• Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished IBCA.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[20,6,"A - Technology & Industrial Base","A.1 - Industrial Base","Is there industrial capability in place that can support the manufacturing of (technology) development articles?","Purpose: \nAssure a thorough assessment of sources for materials and processes that apply to Critical Technologies (CTs) has been completed and that viable sources and potential alternatives have been identified. Assuring that the capability of the defense industrial base to support production of system and subsystem designs is crucial for Milestone B decision makers and should be an exit criterion for PDR.\n","Additional Considerations:\n• Have the majority of manufacturing processes been defined and characterized?\n• Are initial manufacturing approaches documented?\n• Have producibility assessments of key technologies been completed?\n• Has a production cost model been constructed?\n• Can the industrial base support production of development articles?\n• Have long-lead and key supply chain elements been identified? \n• What decisions will impact a supplier's future viability (jeopardize future competitiveness or do not provide a sufficient business case to keep the capabilities/unit around for the future)?\n• What is the certainty of future demand for key products given an economic turndown?\n• What decisions will establish new industrial capabilities (develop new facilities, demonstrate and \"productionize\" new technologies, preserve health of the industrial base)?","Lessons Learned: \n• As designs change, Industrial Base Capability Analyses need to be constantly updated.\n• Failure to consider Industrial base capability as part of the PDR may lead to jeopardizing future production schedules.\n• Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n• Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished IBCA.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[21,6,"A - Technology & Industrial Base","A.1 - Industrial Base","Are the plans to minimize sole, single, and/or foreign sources complete?","Purpose: \nAssure a thorough assessment of sources for materials and processes that apply to Critical Technologies (CTs) has been completed and that viable sources and potential alternatives have been identified. Assuring that the capability of the defense industrial base to support production of system and subsystem designs is crucial for Milestone B decision makers and should be an exit criterion for PDR.\n","Additional Considerations:\n• Have the majority of manufacturing processes been defined and characterized?\n• Are initial manufacturing approaches documented?\n• Have producibility assessments of key technologies been completed?\n• Has a production cost model been constructed?\n• Can the industrial base support production of development articles?\n• Have long-lead and key supply chain elements been identified? \n• What decisions will impact a supplier's future viability (jeopardize future competitiveness or do not provide a sufficient business case to keep the capabilities/unit around for the future)?\n• What is the certainty of future demand for key products given an economic turndown?\n• What decisions will establish new industrial capabilities (develop new facilities, demonstrate and \"productionize\" new technologies, preserve health of the industrial base)?","Lessons Learned: \n• As designs change, Industrial Base Capability Analyses need to be constantly updated.\n• Failure to consider Industrial base capability as part of the PDR may lead to jeopardizing future production schedules.\n• Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n• Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished IBCA.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[22,6,"A - Technology & Industrial Base","A.1 - Industrial Base","Are the plans to minimize potential obsolescence issues complete? ","Purpose: \nAssure a thorough assessment of sources for materials and processes that apply to Critical Technologies (CTs) has been completed and that viable sources and potential alternatives have been identified. Assuring that the capability of the defense industrial base to support production of system and subsystem designs is crucial for Milestone B decision makers and should be an exit criterion for PDR.\n","Additional Considerations:\n• Have the majority of manufacturing processes been defined and characterized?\n• Are initial manufacturing approaches documented?\n• Have producibility assessments of key technologies been completed?\n• Has a production cost model been constructed?\n• Can the industrial base support production of development articles?\n• Have long-lead and key supply chain elements been identified? \n• What decisions will impact a supplier's future viability (jeopardize future competitiveness or do not provide a sufficient business case to keep the capabilities/unit around for the future)?\n• What is the certainty of future demand for key products given an economic turndown?\n• What decisions will establish new industrial capabilities (develop new facilities, demonstrate and \"productionize\" new technologies, preserve health of the industrial base)?","Lessons Learned: \n• As designs change, Industrial Base Capability Analyses need to be constantly updated.\n• Failure to consider Industrial base capability as part of the PDR may lead to jeopardizing future production schedules.\n• Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n• Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished IBCA.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[23,6,"A - Technology & Industrial Base","A.1 - Industrial Base","Has the need for sole, single, and/or foreign sources been justified?"]
[24,6,"A - Technology & Industrial Base","A.1 - Industrial Base","Have potential alternative sources been identified?","Purpose: \nAssure a thorough assessment of sources for materials and processes that apply to Critical Technologies (CTs) has been completed and that viable sources and potential alternatives have been identified. Assuring that the capability of the defense industrial base to support production of system and subsystem designs is crucial for Milestone B decision makers and should be an exit criterion for PDR.\n","Additional Considerations:\n• Have the majority of manufacturing processes been defined and characterized?\n• Are initial manufacturing approaches documented?\n• Have producibility assessments of key technologies been completed?\n• Has a production cost model been constructed?\n• Can the industrial base support production of development articles?\n• Have long-lead and key supply chain elements been identified? \n• What decisions will impact a supplier's future viability (jeopardize future competitiveness or do not provide a sufficient business case to keep the capabilities/unit around for the future)?\n• What is the certainty of future demand for key products given an economic turndown?\n• What decisions will establish new industrial capabilities (develop new facilities, demonstrate and \"productionize\" new technologies, preserve health of the industrial base)?","Lessons Learned: \n• As designs change, Industrial Base Capability Analyses need to be constantly updated.\n• Failure to consider Industrial base capability as part of the PDR may lead to jeopardizing future production schedules.\n• Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n• Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished IBCA.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[25,7,"A - Technology & Industrial Base","A.1 - Industrial Base","Has the industrial capability needed to support production been analyzed?","Purpose: \nAssure the industrial base is stable and viable sources and that potential alternatives have been identified. Assuring that the defense industrial base is stable to support production of system designs should be an exit criterion for CDR.","Additional Considerations:\n• Have the critical manufacturing processes that affect the key characteristics been identified and their capability to meet design tolerances determined?\n• Are materials and tooling available to meet pilot line schedule?\n• Has the system production cost model been updated, allocated to subsystem level, and tracked against targets?\n• Are long-lead procurement plans in place and has the supply chain been assessed?","Lessons Learned: \n• As designs change, Industrial Base Capability Analyses need to be constantly updated.\n• Failure to consider Industrial base capability as part of the CDR may lead to jeopardizing future production schedules.\n• Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n• Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished IBCA.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[26,7,"A - Technology & Industrial Base","A.1 - Industrial Base","Have all sole, single, and foreign sources been assessed?","Purpose: \nAssure the industrial base is stable and viable sources and that potential alternatives have been identified. Assuring that the defense industrial base is stable to support production of system designs should be an exit criterion for CDR.","Additional Considerations:\n• Have the critical manufacturing processes that affect the key characteristics been identified and their capability to meet design tolerances determined?\n• Are materials and tooling available to meet pilot line schedule?\n• Has the system production cost model been updated, allocated to subsystem level, and tracked against targets?\n• Are long-lead procurement plans in place and has the supply chain been assessed?","Lessons Learned: \n• As designs change, Industrial Base Capability Analyses need to be constantly updated.\n• Failure to consider Industrial base capability as part of the CDR may lead to jeopardizing future production schedules.\n• Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n• Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished IBCA.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[27,7,"A - Technology & Industrial Base","A.1 - Industrial Base","Is the stability of all sole, single, and foreign sources being monitored?","Purpose: \nAssure the industrial base is stable and viable sources and that potential alternatives have been identified. Assuring that the defense industrial base is stable to support production of system designs should be an exit criterion for CDR.","Additional Considerations:\n• Have the critical manufacturing processes that affect the key characteristics been identified and their capability to meet design tolerances determined?\n• Are materials and tooling available to meet pilot line schedule?\n• Has the system production cost model been updated, allocated to subsystem level, and tracked against targets?\n• Are long-lead procurement plans in place and has the supply chain been assessed?","Lessons Learned: \n• As designs change, Industrial Base Capability Analyses need to be constantly updated.\n• Failure to consider Industrial base capability as part of the CDR may lead to jeopardizing future production schedules.\n• Key program acquisition decisions and/or the timing thereof may have a major impact on a firm’s ability to support a given program.\n• Program delays or a hiatus in a program acquisition may completely invalidate the results of a previously accomplished IBCA.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[28,7,"A - Technology & Industrial Base","A.1 - Industrial Base","If analyses of industrial capability, sources, and obsolescence indicate risks or issues, have potential alternate sources been developed?"]
[29,8,"A - Technology & Industrial Base","A.1 - Industrial Base","Is an assessment of industrial base capabilities to support the Milestone C decision complete?","Purpose: \nTo demonstrate that the industrial base capabilities required to support the program have been analyzed and assessed to be adequate to achieve LRIP objectives. Approaching a MS C decision, IB considerations are now focused on the prime integrating contractor and supply chain team for production, with consideration of alternate sources required to mitigate specific risks. A “deep dive” analysis can provide the Program Manager with information on issues at the lower tiers of the industrial base that might not otherwise be discovered. This information is necessary to determine whether the program has a stable subcontractor base and assured availability of materials for all critical items.","Additional Considerations: \n• Does the Industrial Capabilities Assessment (IBCA) include an analysis of anticipated Subcontractors, Suppliers and Vendors?\n• Have all subcontractors/suppliers been identified? Are back-ups available for all critical items?\n• Have all Sole/Single/Foreign Sources been identified and have alternative plans been made for parts available from foreign sources? Are these sources being monitored for stability?\n• Are industrial capabilities and capacity adequate to support Program needs?\n• Have contingency program requirements been validated and documented?\n• Have Long-Lead items been analyzed?\n• Are plans in effect to mitigate potential program schedule impacts?\n• Has multi-sourcing been employed where possible?\n• Have delivery schedules been demonstrated to support the LRIP build?• Have impacts of this acquisition program on the industrial base been analyzed?\n• Does the IBCA address industrial Sector, and Individual Enterprise trends?\n• Does the IBCA address environmental impacts?\n• Does the IBCA address potential program stretch-out and re-start challenges?\n• Has the competition plan been updated?\n• Have co-development/co-production opportunities been updated?\n• Are there any proprietary processes that could impact government data rights?","Lessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier to define the supply chain/industrial base for the program, with particular emphasis on critical/pacing suppliers and long-lead items. \n• Earlier IBCAs for the program should have identified broader industrial base issues, such as sector trends and alternate sources of parts and material. At this point in the program, the IBCA should focus on the financial viability of the contractor team/supply chain in-place going into production.\n• At the higher levels of the industrial chain, key issues include knowledge of technology/ product refresh cycles and supplier turnover (DMSMS/obsolescence issues). \n• At the lower levels, where the industrial base often necks down to a single source, the IBCA is often the best way to identify potential risks (such as changes environmental laws impacting the product/process) before they become problems.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[30,8,"A - Technology & Industrial Base","A.1 - Industrial Base","Does analysis show that industrial capabilities are in place to support Low Rate (Initial) Production?","Purpose: \nTo demonstrate that the industrial base capabilities required to support the program have been analyzed and assessed to be adequate to achieve LRIP objectives. Approaching a MS C decision, IB considerations are now focused on the prime integrating contractor and supply chain team for production, with consideration of alternate sources required to mitigate specific risks. A “deep dive” analysis can provide the Program Manager with information on issues at the lower tiers of the industrial base that might not otherwise be discovered. This information is necessary to determine whether the program has a stable subcontractor base and assured availability of materials for all critical items.","Additional Considerations: \n• Does the Industrial Capabilities Assessment (IBCA) include an analysis of anticipated Subcontractors, Suppliers and Vendors?\n• Have all subcontractors/suppliers been identified? Are back-ups available for all critical items?\n• Have all Sole/Single/Foreign Sources been identified and have alternative plans been made for parts available from foreign sources? Are these sources being monitored for stability?\n• Are industrial capabilities and capacity adequate to support Program needs?\n• Have contingency program requirements been validated and documented?\n• Have Long-Lead items been analyzed?\n• Are plans in effect to mitigate potential program schedule impacts?\n• Has multi-sourcing been employed where possible?\n• Have delivery schedules been demonstrated to support the LRIP build?• Have impacts of this acquisition program on the industrial base been analyzed?\n• Does the IBCA address industrial Sector, and Individual Enterprise trends?\n• Does the IBCA address environmental impacts?\n• Does the IBCA address potential program stretch-out and re-start challenges?\n• Has the competition plan been updated?\n• Have co-development/co-production opportunities been updated?\n• Are there any proprietary processes that could impact government data rights?","Lessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier to define the supply chain/industrial base for the program, with particular emphasis on critical/pacing suppliers and long-lead items. \n• Earlier IBCAs for the program should have identified broader industrial base issues, such as sector trends and alternate sources of parts and material. At this point in the program, the IBCA should focus on the financial viability of the contractor team/supply chain in-place going into production.\n• At the higher levels of the industrial chain, key issues include knowledge of technology/ product refresh cycles and supplier turnover (DMSMS/obsolescence issues). \n• At the lower levels, where the industrial base often necks down to a single source, the IBCA is often the best way to identify potential risks (such as changes environmental laws impacting the product/process) before they become problems.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[31,8,"A - Technology & Industrial Base","A.1 - Industrial Base","Are sources available?","Purpose: \nTo demonstrate that the industrial base capabilities required to support the program have been analyzed and assessed to be adequate to achieve LRIP objectives. Approaching a MS C decision, IB considerations are now focused on the prime integrating contractor and supply chain team for production, with consideration of alternate sources required to mitigate specific risks. A “deep dive” analysis can provide the Program Manager with information on issues at the lower tiers of the industrial base that might not otherwise be discovered. This information is necessary to determine whether the program has a stable subcontractor base and assured availability of materials for all critical items.","Additional Considerations: \n• Does the Industrial Capabilities Assessment (IBCA) include an analysis of anticipated Subcontractors, Suppliers and Vendors?\n• Have all subcontractors/suppliers been identified? Are back-ups available for all critical items?\n• Have all Sole/Single/Foreign Sources been identified and have alternative plans been made for parts available from foreign sources? Are these sources being monitored for stability?\n• Are industrial capabilities and capacity adequate to support Program needs?\n• Have contingency program requirements been validated and documented?\n• Have Long-Lead items been analyzed?\n• Are plans in effect to mitigate potential program schedule impacts?\n• Has multi-sourcing been employed where possible?\n• Have delivery schedules been demonstrated to support the LRIP build?• Have impacts of this acquisition program on the industrial base been analyzed?\n• Does the IBCA address industrial Sector, and Individual Enterprise trends?\n• Does the IBCA address environmental impacts?\n• Does the IBCA address potential program stretch-out and re-start challenges?\n• Has the competition plan been updated?\n• Have co-development/co-production opportunities been updated?\n• Are there any proprietary processes that could impact government data rights?","Lessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier to define the supply chain/industrial base for the program, with particular emphasis on critical/pacing suppliers and long-lead items. \n• Earlier IBCAs for the program should have identified broader industrial base issues, such as sector trends and alternate sources of parts and material. At this point in the program, the IBCA should focus on the financial viability of the contractor team/supply chain in-place going into production.\n• At the higher levels of the industrial chain, key issues include knowledge of technology/ product refresh cycles and supplier turnover (DMSMS/obsolescence issues). \n• At the lower levels, where the industrial base often necks down to a single source, the IBCA is often the best way to identify potential risks (such as changes environmental laws impacting the product/process) before they become problems.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[32,8,"A - Technology & Industrial Base","A.1 - Industrial Base","When cost-effective or needed to mitigate risks, are multiple sources included?"]
[33,9,"A - Technology & Industrial Base","A.1 - Industrial Base","Is an assessment of industrial base capabilities to support the Full Rate Production (FRP) decision complete?","Purpose: \nThe intent is to demonstrate that the industrial base capabilities required to support the program have been analyzed and assessed to be adequate to achieve FRP objectives. After the MS C decision, IB considerations are now focused on the prime integrating contractor and supply chain team for full rate production, with consideration of alternate sources required to mitigate specific risks. \n\n\n","Additional Considerations: \n• Have all subcontractors/suppliers been identified? Are back-ups available for all critical items?\n• Are Sole/Single/Foreign Sources being monitored for stability?\n• Are industrial capabilities and capacity adequate to support program needs?\n• Have contingency program requirements been validated and documented?\n• Are plans in effect to mitigate potential program schedule impacts?\n• Has multi-sourcing been employed where possible?\n• Have delivery schedules been demonstrated to support the FRP build?\n• Have impacts of this acquisition program on the industrial base been analyzed?\n• Does the IBCA address environmental impacts?\n• Does the IBCA address potential program stretch-out and re-start challenges?","Lessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier to define the supply chain/industrial base for the program, with particular emphasis on critical/pacing suppliers and long-lead items. \n• Earlier IBCAs for the program should have identified broader industrial base issues, such as sector trends and alternate sources of parts and material. In this regard, at this point in the program, the IBCA should focus on the financial viability of the contractor team/supply chain in-place going into production.\n• At the higher levels of the industrial chain, key issues include knowledge of technology/ product refresh cycles and supplier turnover (DMSMS/obsolescence issues). \n• At the lower levels, where the industrial base often necks down to a single source, the IBCA is often the best way to identify potential risks (such as changes environmental laws impacting the product/process) before they become problems.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[34,9,"A - Technology & Industrial Base","A.1 - Industrial Base","Is the Industrial capability in place to support the start of Full Rate Production (FRP)?","Purpose: \nThe intent is to demonstrate that the industrial base capabilities required to support the program have been analyzed and assessed to be adequate to achieve FRP objectives. After the MS C decision, IB considerations are now focused on the prime integrating contractor and supply chain team for full rate production, with consideration of alternate sources required to mitigate specific risks. \n\n\n","Additional Considerations: \n• Have all subcontractors/suppliers been identified? Are back-ups available for all critical items?\n• Are Sole/Single/Foreign Sources being monitored for stability?\n• Are industrial capabilities and capacity adequate to support program needs?\n• Have contingency program requirements been validated and documented?\n• Are plans in effect to mitigate potential program schedule impacts?\n• Has multi-sourcing been employed where possible?\n• Have delivery schedules been demonstrated to support the FRP build?\n• Have impacts of this acquisition program on the industrial base been analyzed?\n• Does the IBCA address environmental impacts?\n• Does the IBCA address potential program stretch-out and re-start challenges?","Lessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier to define the supply chain/industrial base for the program, with particular emphasis on critical/pacing suppliers and long-lead items. \n• Earlier IBCAs for the program should have identified broader industrial base issues, such as sector trends and alternate sources of parts and material. In this regard, at this point in the program, the IBCA should focus on the financial viability of the contractor team/supply chain in-place going into production.\n• At the higher levels of the industrial chain, key issues include knowledge of technology/ product refresh cycles and supplier turnover (DMSMS/obsolescence issues). \n• At the lower levels, where the industrial base often necks down to a single source, the IBCA is often the best way to identify potential risks (such as changes environmental laws impacting the product/process) before they become problems.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[35,10,"A - Technology & Industrial Base","A.1 - Industrial Base","Does analysis show that industrial capabilities are supporting Full Rate Production (FRP)?","Purpose: \nThe intent is to demonstrate that the industrial base capabilities required to support the program have been analyzed and continue to be adequate to support FRP. After the FRP decision, IB considerations are now focused on the capability of the industrial base to continue to support full rate production, including modifications, upgrades, surge, and other potential manufacturing requirements.\n\n\n","Additional Considerations: \n• Have all subcontractors/suppliers been identified? Are back-ups available for all critical items?\n• Are Sole/Single/Foreign Sources being monitored for stability?\n• Are industrial capabilities and capacity adequate to support program needs?\n• Have contingency program requirements been validated and documented?\n• Are plans in effect to mitigate potential program schedule impacts?\n• Have delivery schedules been demonstrated to support the FRP build?\n• Have impacts of this acquisition program on the industrial base been analyzed?\n","\nLessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier to define the supply chain/industrial base for the program, with particular emphasis on critical/pacing suppliers and long-lead items. \n• Earlier IBCAs for the program should have identified broader industrial base issues, such as sector trends and alternate sources of parts and material. In this regard, at this point in the program, the IBCA should focus on the financial viability of the contractor team/supply chain in-place going into production.\n• At the higher levels of the industrial chain, key issues include knowledge of technology/ product refresh cycles and supplier turnover (DMSMS/obsolescence issues). \n• At the lower levels, where the industrial base often necks down to a single source, the IBCA is often the best way to identify potential risks (such as changes environmental laws impacting the product/process) before they become problems.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[36,10,"A - Technology & Industrial Base","A.1 - Industrial Base","Has the industrial capability been assessed for capability to support modifications, upgrades, surge quantities, and other potential manufacturing requirements?","Purpose: \nThe intent is to demonstrate that the industrial base capabilities required to support the program have been analyzed and continue to be adequate to support FRP. After the FRP decision, IB considerations are now focused on the capability of the industrial base to continue to support full rate production, including modifications, upgrades, surge, and other potential manufacturing requirements.\n\n\n","Additional Considerations: \n• Have all subcontractors/suppliers been identified? Are back-ups available for all critical items?\n• Are Sole/Single/Foreign Sources being monitored for stability?\n• Are industrial capabilities and capacity adequate to support program needs?\n• Have contingency program requirements been validated and documented?\n• Are plans in effect to mitigate potential program schedule impacts?\n• Have delivery schedules been demonstrated to support the FRP build?\n• Have impacts of this acquisition program on the industrial base been analyzed?\n","\nLessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier to define the supply chain/industrial base for the program, with particular emphasis on critical/pacing suppliers and long-lead items. \n• Earlier IBCAs for the program should have identified broader industrial base issues, such as sector trends and alternate sources of parts and material. In this regard, at this point in the program, the IBCA should focus on the financial viability of the contractor team/supply chain in-place going into production.\n• At the higher levels of the industrial chain, key issues include knowledge of technology/ product refresh cycles and supplier turnover (DMSMS/obsolescence issues). \n• At the lower levels, where the industrial base often necks down to a single source, the IBCA is often the best way to identify potential risks (such as changes environmental laws impacting the product/process) before they become problems.\n• Fragility of the IB and criticality of critical components need to be considered in the IBCA."]
[37,1,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Have global trends in manufacturing science and technology been identified (i.e., concepts, capabilities)?","Purpose: \nTo begin identify new manufacturing requirements (e.g. concepts) to build future products and to use this information to explore potential solutions. This information will give decision makers insight into the risk and requirements of new manufacturing concepts and will allow them to explore potential solutions for concepts early in the S&T phase. The S&T community needs to have a basic understanding on the manufacturing requirements and the risk and implications to proceed with the proposed approach.","Additional Considerations: \n• Has manufacturing risk (i.e., availability of process, sources and materials) been considered?","Lessons Learned: \n• Historically we have not adequately addressed the manufacturing risk in this phase of technology development, so we must start addressing these risk issues earlier in the system life cycle."]
[38,2,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Have potential manufacturing science and technology gaps been identified?","Purpose: \nTo begin identifying manufacturing concepts to produce the product via experimentation and models and to use this information to understand the risk in pursuing these concepts. This will give decision makers insight into the risk and requirements for the new materials and processes and allow them to explore potential solutions for new concepts early in the S&T phase. To effectively transition technology programs, one must address the manufacturing maturity of the prototypes being developed. Assessing the maturity of the manufacturing processes in S&T mandates that the R&D and acquisition communities work together, and a key part of that assessment is understanding manufacturing risk drivers and implications to production development strategy.","Additional Considerations: \n• Has manufacturing risk (i.e., availability of process, sources and materials) been considered?\n","Lessons Learned: \n• Historically we have not adequately addressed the manufacturing risk in this phase of technology development, so we must start addressing these risk issues earlier in the system life cycle."]
[39,3,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Have Manufacturing technology requirements been identified to address potential capability gaps for system concepts?","Purpose: \nTo start assessing the manufacturing processes of the proposed alternatives so as to identify advanced manufacturing technologies required to support those alternative designs. To use this process information to understand risk of the various alternatives as the program proceeds into Technology Development (TD). To quantify the risk of achieving program objectives and implement mitigation efforts. The program needs to understand the manufacturing risk of each proposed alternative and to reflect that risk in program planning efforts. It is crucial to perform a manufacturing feasibility study each proposed alternative and understand what, if any, new manufacturing technologies are required. At the end of this phase, the alternative that best addresses user requirements will proceed into TD. Manufacturing risk of selected alternative(s) will also be addressed in funding, planning, and other key program documentations (e.g., the Technology Development Strategy).","Additional Considerations: \n• Has manufacturing risk (i.e., availability of process, sources and materials) been considered?\n","Lessons Learned: \n• Historically we have not adequately addressed the manufacturing risk in this phase of technology development, so we must start addressing these risk issues earlier in the system life cycle."]
[40,4,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Have manufacturing technology development initiatives been defined for preferred materiel solution?","Purpose: \nTo assess process capability by using data from prototype builds and similar production processes to assess if target costs are achievable. To determine whether manufacturing development efforts or new technologies are required to achieve program objectives. Using actual production data when performing these assessments can ensure that manufacturing risk of transitioning into production has been adequately assessed. At the end of this phase the best alternatives will be selected to proceed into EMD. Risk will also be understood for processes that require manufacturing technology development. \n","Additional Considerations: \n• Has manufacturing risk (i.e., availability of process, sources and materials) been considered?\n• The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms.\n","Lessons Learned: \n• Historically we have not adequately addressed the manufacturing risk in this phase of technology development, so we must start addressing these risk issues earlier in the system life cycle."]
[41,4,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Are manufacturing technology development requirements considered in the AoA?","Purpose: \nTo assess the progress of on-going manufacturing technology efforts and, using data generated in a production relevant environment, to determine whether new efforts are required. The program manager must understand the necessary manufacturing technology","Additional Considerations: \n• Are processes for fabrication of new or exotic materials clearly understood and available, or are they under development at this stage?\n• The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms.","Lessons Learned: \n• To adequately manage risk one must understand the progress of manufacturing technology development as the program enters EMD."]
[42,5,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Have the required manufacturing technology development efforts been initiated as applicable?","Purpose: \nTo assess the progress of on-going manufacturing technology efforts and, using data generated in a production relevant environment, to determine whether new efforts are required. The program manager must understand the necessary manufacturing technology","Additional Considerations: \n• Are processes for fabrication of new or exotic materials clearly understood and available, or are they under development at this stage?\n• The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms.","Lessons Learned: \n• To adequately manage risk one must understand the progress of manufacturing technology development as the program enters EMD."]
[43,6,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Are the necessary manufacturing technology development efforts underway and continuing?","Purpose: \nTo ensure that adequate progress in our manufacturing technology efforts are being made by using demonstrated results from a production representative environment. The program needs to verify that the needed manufacturing technology efforts are being supported and progressing satisfactorily. It is essential to have confidence in the program’s ability to provide required manufacturing technology to support program objectives.","Additional Considerations: \n• Are processes for fabrication of new or exotic materials clearly understood and available, or are they under development at this stage?\n• The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms.","Lessons Learned: \n• To adequately manage risk one must understand the progress of manufacturing technology development as the program progresses in EMD."]
[44,6,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Have the manufacturing technology development solutions required been successfully demonstrated in a production relevant environment?","Purpose: \nTo ensure that adequate progress in our manufacturing technology efforts are being made by using demonstrated results from a production representative environment. The program needs to verify that the needed manufacturing technology efforts are being supported and progressing satisfactorily. It is essential to have confidence in the program’s ability to provide required manufacturing technology to support program objectives.","Additional Considerations: \n• Are processes for fabrication of new or exotic materials clearly understood and available, or are they under development at this stage?\n• The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms.","Lessons Learned: \n• To adequately manage risk one must understand the progress of manufacturing technology development as the program progresses in EMD."]
[45,7,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Are manufacturing technology efforts continuing and on schedule for pilot line implementation?","Purpose: \nTo ensure that adequate progress in the manufacturing technology efforts is being made by using demonstrated results from a pilot line environment. To demonstrate that equipment and processes utilize mature technology and are adequate to achieve Low Rate Initial Production (LRIP) objectives. Manufacturing equipment and processes should be based on new technology that is at a TRL of 8 or higher to support a pilot line for LRIP. The program must verify that all manufacturing technology efforts are supported and progressing satisfactory. This will ensure a that the production yield is sufficient to meet the program development schedule and that rework does not impact program cost and schedule. A Pilot Line provides verifiable process capability/ performance results to compare with LRIP requirements. Some of the data is used to determine whether tolerances can be met for Key Characteristics. The program manager can then determine the likelihood of achieving the process capability to meet LRIP requirements and what risks remain going into LRIP.","Additional Considerations: \n• How faithful is the pilot line to a true production environment (personnel, equipment, processes, work instructions, etc.)?","Lessons Learned: \n• One of the important steps for assuring that adequate improvement in needed manufacturing technology is being made is to work with the contractor/ supplier on what truly represents a pilot production line for your programs. This effort should be done as soon as practical and if at all possible, before the award of the contract where this activity will be performed (e.g. EMD). \n• All ACAT programs are required to conduct a Technology Readiness Assessment (TRA) to determine if new technology is being used in the system design. Manufacturing technology must be considered as part of the TRA. Reviewing TRA results can provide indication that new manufacturing technology is being used. The TRA may miss new manufacturing technology however, since TRAs tend to focus on system design. \n• A review of information collected from earlier MRL assessments and findings is another important source of identifying new manufacturing technology insertion. \n• The third and final way is to review each the pilot production lines for all parts of system development to determine if any new equipment or processes are being used. For introduction of new equipment or processes, one must determine if it is based on introduction of new technology or reengineering of existing manufacturing systems—not based on new technology. \n• A complete audit of the pilot production line equipment and processes can identify new technology being inserted. \n• Once identified, focusing in on specific Statistical Process Control (SPC) measures may not be possible due to lot sizes (i.e., minimal data available), but other performance output measures can also be used to determine process capability (e.g., First Pass Yield, Scrap/Rework, Hours, Cycle Time, etc.). \n• Results that fall into expected norms can indicate that new technology is performing up to expectations, but results out of the expected band indicate possible influence of immature manufacturing technology and require a closer look. \n• Root cause analysis of negative results and unexpected outputs may also indicate immature manufacturing technology."]
[46,7,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Have the required manufacturing science and manufacturing technology development solutions been successfully demonstrated in a production representative environment?","Purpose: \nTo ensure that adequate progress in the manufacturing technology efforts is being made by using demonstrated results from a pilot line environment. To demonstrate that equipment and processes utilize mature technology and are adequate to achieve Low Rate Initial Production (LRIP) objectives. Manufacturing equipment and processes should be based on new technology that is at a TRL of 8 or higher to support a pilot line for LRIP. The program must verify that all manufacturing technology efforts are supported and progressing satisfactory. This will ensure a that the production yield is sufficient to meet the program development schedule and that rework does not impact program cost and schedule. A Pilot Line provides verifiable process capability/ performance results to compare with LRIP requirements. Some of the data is used to determine whether tolerances can be met for Key Characteristics. The program manager can then determine the likelihood of achieving the process capability to meet LRIP requirements and what risks remain going into LRIP.","Additional Considerations: \n• How faithful is the pilot line to a true production environment (personnel, equipment, processes, work instructions, etc.)?","Lessons Learned: \n• One of the important steps for assuring that adequate improvement in needed manufacturing technology is being made is to work with the contractor/ supplier on what truly represents a pilot production line for your programs. This effort should be done as soon as practical and if at all possible, before the award of the contract where this activity will be performed (e.g. EMD). \n• All ACAT programs are required to conduct a Technology Readiness Assessment (TRA) to determine if new technology is being used in the system design. Manufacturing technology must be considered as part of the TRA. Reviewing TRA results can provide indication that new manufacturing technology is being used. The TRA may miss new manufacturing technology however, since TRAs tend to focus on system design. \n• A review of information collected from earlier MRL assessments and findings is another important source of identifying new manufacturing technology insertion. \n• The third and final way is to review each the pilot production lines for all parts of system development to determine if any new equipment or processes are being used. For introduction of new equipment or processes, one must determine if it is based on introduction of new technology or reengineering of existing manufacturing systems—not based on new technology. \n• A complete audit of the pilot production line equipment and processes can identify new technology being inserted. \n• Once identified, focusing in on specific Statistical Process Control (SPC) measures may not be possible due to lot sizes (i.e., minimal data available), but other performance output measures can also be used to determine process capability (e.g., First Pass Yield, Scrap/Rework, Hours, Cycle Time, etc.). \n• Results that fall into expected norms can indicate that new technology is performing up to expectations, but results out of the expected band indicate possible influence of immature manufacturing technology and require a closer look. \n• Root cause analysis of negative results and unexpected outputs may also indicate immature manufacturing technology."]
[47,8,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Are the primary manufacturing technology efforts concluding? "]
[48,8,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Are manufacturing improvement efforts continuing?","Purpose: \nTo assess whether manufacturing technologies efforts are required to meet Full Rate Production (FRP) requirements, and if so, to ensure they are initiated. The program constantly needs to assess if manufacturing technology efforts are needed to meet the program objectives.","Additional Considerations: \n• Look closely at yields and rates during LRIP to assure that you can effectively ramp up to FRP.","Lessons Learned: \n• Continuously assess production process and capabilities to support program requirements.\n• Implementing manufacturing technologies can be a critical tool for helping overcome shortfalls – especially in cost targets."]
[49,8,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Have the manufacturing technology solutions required been validated on a pilot line?","Purpose: \nTo assess manufacturing technologies to improve efforts for FRP. The program constantly needs to assess options to improve the performance. Implement manufacturing technology programs effectively can be a major tool in this effort.","Additional Considerations: \n• None","\nLessons Learned: \n• A continuous improvement program and resultant elimination of waste (LEAN manufacturing) are critical indicators as to whether implementation of the manufacturing technologies has been truly successful."]
[50,9,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Have manufacturing technology process improvement efforts been initiated for Full Rate Production (FRP)? ","Purpose: \nTo identify manufacturability and producibility for the alternative being considered and to use this information to understand manufacturing risk in managing the transition. To see some high level process charts to support this effort, and for new processes, have some data available based on a minimum of experimental data and modeling. Manufacturability and producibility historically have not been evaluated until the product design is virtually frozen, thus incurring excessive product cost and product/process quality problems. Having to redesign the product to be affordable and capable of being produced in quantity is cost, quality and schedule prohibitive.","Additional Considerations: \n• Is there a need to evaluate/re-evaluate materials/processes characteristics for manufacturability and/or producibility issues? \n• Have new manufacturing concepts and potential solutions been identified that could impact manufacturability/producibility?\n• Have new item gaps/risks impacting manufacturability and/or producibility been identified for key technologies, components, and/or key processes?\n• Have initial estimates of yields and rates based on experiments or state of the art been completed that could show potential manufacturability and/or producibility issue?","Lessons Learned: \n• Historically manufacturability and producibility issues are ignored early in S&T. \n• Producibility and manufacturability risks need to be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives."]
[51,10,"A - Technology & Industrial Base","A.2 - Manufacturing Technology Development","Are manufacturing technology continuous process improvements ongoing?","Purpose:\nTo ensure the producibility and manufacturability assessments have been performed and that the results will be used to identify the risk drivers in manufacturing proposed design alternatives.\nThe program must ensure that producibility and manufacturability risks are being adequately assessed and that the best alternatives will be considered.","Additional Considerations: \n• Is there a need to evaluate/re-evaluate materials/processes characteristics for manufacturability and/or producibility issues? \n• Have new manufacturing concepts and potential solutions been identified that could impact manufacturability/producibility?\n• Have new item gaps/risks impacting manufacturability and/or producibility been identified for key technologies, components, and/or key processes?\n• Have initial estimates of yields and rates based on experiments or state of the art been completed that could show potential manufacturability and/or producibility issue?\n","Lessons Learned: \n• Historically manufacturability and producibility issues are ignored early in S&T. \n• Producibility and manufacturability risks need to be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives."]
[52,1,"B - Design","B.1 - Producibility Program","Have hypotheses been developed for cause-effect relationships between technology variables and producibility?","Purpose:\nTo ensure the producibility and manufacturability assessments have been performed and that the results will be used to identify the risk drivers in manufacturing proposed design alternatives.\nThe program must ensure that producibility and manufacturability risks are being adequately assessed and that the best alternatives will be considered.","Additional Considerations: \n• Is there a need to evaluate/re-evaluate materials/processes characteristics for manufacturability and/or producibility issues? \n• Have new manufacturing concepts and potential solutions been identified that could impact manufacturability/producibility?\n• Have new item gaps/risks impacting manufacturability and/or producibility been identified for key technologies, components, and/or key processes?\n• Have initial estimates of yields and rates based on experiments or state of the art been completed that could show potential manufacturability and/or producibility issue?\n","Lessons Learned: \n• Historically manufacturability and producibility issues are ignored early in S&T. \n• Producibility and manufacturability risks need to be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives."]
[53,2,"B - Design","B.1 - Producibility Program","Have studies been performed to test hypotheses regarding cause-effect relationships between technology variables and producibility? ","Purpose:\nTo ensure the producibility and manufacturability assessments have been performed and that the results will be used to identify the risk drivers in manufacturing proposed design alternatives.\nThe program must ensure that producibility and manufacturability risks are being adequately assessed and that the best alternatives will be considered.","Additional Considerations: \n• Is there a need to evaluate/re-evaluate materials/processes characteristics for manufacturability and/or producibility issues? \n• Have new manufacturing concepts and potential solutions been identified that could impact manufacturability/producibility?\n• Have new item gaps/risks impacting manufacturability and/or producibility been identified for key technologies, components, and/or key processes?\n• Have initial estimates of yields and rates based on experiments or state of the art been completed that could show potential manufacturability and/or producibility issue?\n","Lessons Learned: \n• Historically manufacturability and producibility issues are ignored early in S&T. \n• Producibility and manufacturability risks need to be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives."]
[54,2,"B - Design","B.1 - Producibility Program","Have elements been identified which have a potential impact to producibility (i.e., materials, processes, capabilities, limitations)?","Purpose: \nTo obtain more fidelity in the producibility and manufacturability risk assessments by using prototype build data to focus on risk drivers. At this stage the focus to identify manufacturing constraints should be at the component level for each prototype built, addressing specific producibility and manufacturability concerns (e.g., materials, labor, equipment, tooling/Special Test Equipment, setup, yield/scrap/rework, Work In Progress, and capability/capacity constraints). It is essential for the program to address these issues when performing design trades to minimize risk and cost impact to the program. At the end of this phase the best available alternatives will be selected to proceed into EMD.","Additional Considerations: \n• Have all relevant materials/processes been evaluated for manufacturability and producibility in a production relevant environment?\n• Can properly trained production personnel ultimately repeat the prototype work performed by engineers?\n• Is the tooling and methods used, if applicable, be applicable ultimately in production?\n• What are the ultimate manufacturing cost implications when in production? \n• Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n","Lessons Learned: \n• Historically manufacturability and producibility issues are ignored early in acquisition. \n• Producibility and manufacturability risks of the alternatives need to be presented to the decision makers before down-selecting the preferred alternative design(s) to understand the overall risk of achieving program objectives.\n• As the programs move from Milestone A into the Technology Development Phase there is a tendency for the manufacturing community to be insufficiently involved as a key team member.\n• If Manufacturability and producibility are continuously evaluated as the product technology design matures redesign costs and schedule delays can be avoided."]
[55,3,"B - Design","B.1 - Producibility Program","Have relevant materials/processes been evaluated for manufacturability using experiments/models?","Purpose: \nTo obtain more fidelity in the producibility and manufacturability risk assessments by using prototype build data to focus on risk drivers. At this stage the focus to identify manufacturing constraints should be at the component level for each prototype built, addressing specific producibility and manufacturability concerns (e.g., materials, labor, equipment, tooling/Special Test Equipment, setup, yield/scrap/rework, Work In Progress, and capability/capacity constraints). It is essential for the program to address these issues when performing design trades to minimize risk and cost impact to the program. At the end of this phase the best available alternatives will be selected to proceed into EMD.","Additional Considerations: \n• Have all relevant materials/processes been evaluated for manufacturability and producibility in a production relevant environment?\n• Can properly trained production personnel ultimately repeat the prototype work performed by engineers?\n• Is the tooling and methods used, if applicable, be applicable ultimately in production?\n• What are the ultimate manufacturing cost implications when in production? \n• Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n","Lessons Learned: \n• Historically manufacturability and producibility issues are ignored early in acquisition. \n• Producibility and manufacturability risks of the alternatives need to be presented to the decision makers before down-selecting the preferred alternative design(s) to understand the overall risk of achieving program objectives.\n• As the programs move from Milestone A into the Technology Development Phase there is a tendency for the manufacturing community to be insufficiently involved as a key team member.\n• If Manufacturability and producibility are continuously evaluated as the product technology design matures redesign costs and schedule delays can be avoided."]
[56,4,"B - Design","B.1 - Producibility Program","Are initial producibility and manufacturability assessments in selection of preferred materiel solution complete?","Purpose: \nTo obtain more fidelity in the producibility and manufacturability risk assessments by using prototype build data to focus on risk drivers. At this stage the focus to identify manufacturing constraints should be at the component level for each prototype built, addressing specific producibility and manufacturability concerns (e.g., materials, labor, equipment, tooling/Special Test Equipment, setup, yield/scrap/rework, Work In Progress, and capability/capacity constraints). It is essential for the program to address these issues when performing design trades to minimize risk and cost impact to the program. At the end of this phase the best available alternatives will be selected to proceed into EMD.","Additional Considerations: \n• Have all relevant materials/processes been evaluated for manufacturability and producibility in a production relevant environment?\n• Can properly trained production personnel ultimately repeat the prototype work performed by engineers?\n• Is the tooling and methods used, if applicable, be applicable ultimately in production?\n• What are the ultimate manufacturing cost implications when in production? \n• Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n","Lessons Learned: \n• Historically manufacturability and producibility issues are ignored early in acquisition. \n• Producibility and manufacturability risks of the alternatives need to be presented to the decision makers before down-selecting the preferred alternative design(s) to understand the overall risk of achieving program objectives.\n• As the programs move from Milestone A into the Technology Development Phase there is a tendency for the manufacturing community to be insufficiently involved as a key team member.\n• If Manufacturability and producibility are continuously evaluated as the product technology design matures redesign costs and schedule delays can be avoided."]
[57,4,"B - Design","B.1 - Producibility Program","Are initial producibility and manufacturability assessments in selection of preferred materiel solution been considered in AoA ? "]
[58,4,"B - Design","B.1 - Producibility Program","Are initial producibility and manufacturability assessments in selection of preferred materiel solution been documented in AS key components/technologies? ","Purpose:\nTo assess producibility data from building prototypes in a production relevant environment, compare it to targets and assess manufacturing risk of meeting future requirements. To use this data to ensure that the Acquisition Strategy, Systems Engineering Plan, Manufacturing and Producibility plans, and planning for EMD apply appropriate risk mitigation and make appropriate trades in the preliminary design process. At this phase of the program the design must be produced in a production-relevant environment, and producibility results must be used in design trade-offs and program planning efforts. This is the opportune time to make design changes without negatively impacting program cost and schedule. Addressing producibility at this stage of the acquisition cycle is critical to achieving affordability objectives.","Additional Considerations: \n• Are the tooling and methods used applicable ultimately in production?\n• Have the manufacturing processes been assessed for capability to test and verify in production and their influence on Operations and Support (O&S)?\n• Have all relevant materials/processes been evaluated for manufacturability and producibility in a production relevant environment? \n• Can properly trained production personnel repeat the prototype work performed by engineers? \n• Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n• What are the ultimate manufacturing cost implications when in production?","Lessons Learned:\n• Historically manufacturability and producibility issues are ignored early in acquisition. \n• Producibility and manufacturability risks of the alternatives need to be presented to the decision makers before moving into EMD.\n• This is the opportune time to make design changes without negatively impacting program cost and schedule."]
[59,5,"B - Design","B.1 - Producibility Program","Have producibility & manufacturability assessments of key technologies and components been initiated?","Purpose:\nTo assess producibility data from building prototypes in a production relevant environment, compare it to targets and assess manufacturing risk of meeting future requirements. To use this data to ensure that the Acquisition Strategy, Systems Engineering Plan, Manufacturing and Producibility plans, and planning for EMD apply appropriate risk mitigation and make appropriate trades in the preliminary design process. At this phase of the program the design must be produced in a production-relevant environment, and producibility results must be used in design trade-offs and program planning efforts. This is the opportune time to make design changes without negatively impacting program cost and schedule. Addressing producibility at this stage of the acquisition cycle is critical to achieving affordability objectives.","Additional Considerations: \n• Are the tooling and methods used applicable ultimately in production?\n• Have the manufacturing processes been assessed for capability to test and verify in production and their influence on Operations and Support (O&S)?\n• Have all relevant materials/processes been evaluated for manufacturability and producibility in a production relevant environment? \n• Can properly trained production personnel repeat the prototype work performed by engineers? \n• Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n• What are the ultimate manufacturing cost implications when in production?","Lessons Learned:\n• Historically manufacturability and producibility issues are ignored early in acquisition. \n• Producibility and manufacturability risks of the alternatives need to be presented to the decision makers before moving into EMD.\n• This is the opportune time to make design changes without negatively impacting program cost and schedule."]
[60,5,"B - Design","B.1 - Producibility Program","Do the ongoing design trades include consideration of manufacturing processes and industrial base capability constraints? ","Purpose:\nTo assess producibility data from building prototypes in a production relevant environment, compare it to targets and assess manufacturing risk of meeting future requirements. To use this data to ensure that the Acquisition Strategy, Systems Engineering Plan, Manufacturing and Producibility plans, and planning for EMD apply appropriate risk mitigation and make appropriate trades in the preliminary design process. At this phase of the program the design must be produced in a production-relevant environment, and producibility results must be used in design trade-offs and program planning efforts. This is the opportune time to make design changes without negatively impacting program cost and schedule. Addressing producibility at this stage of the acquisition cycle is critical to achieving affordability objectives.","Additional Considerations: \n• Are the tooling and methods used applicable ultimately in production?\n• Have the manufacturing processes been assessed for capability to test and verify in production and their influence on Operations and Support (O&S)?\n• Have all relevant materials/processes been evaluated for manufacturability and producibility in a production relevant environment? \n• Can properly trained production personnel repeat the prototype work performed by engineers? \n• Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n• What are the ultimate manufacturing cost implications when in production?","Lessons Learned:\n• Historically manufacturability and producibility issues are ignored early in acquisition. \n• Producibility and manufacturability risks of the alternatives need to be presented to the decision makers before moving into EMD.\n• This is the opportune time to make design changes without negatively impacting program cost and schedule."]
[61,5,"B - Design","B.1 - Producibility Program","Have the manufacturing processes been assessed for capability to be tested and verified in production? ","Purpose:\nTo assess producibility data from building prototypes in a production relevant environment, compare it to targets and assess manufacturing risk of meeting future requirements. To use this data to ensure that the Acquisition Strategy, Systems Engineering Plan, Manufacturing and Producibility plans, and planning for EMD apply appropriate risk mitigation and make appropriate trades in the preliminary design process. At this phase of the program the design must be produced in a production-relevant environment, and producibility results must be used in design trade-offs and program planning efforts. This is the opportune time to make design changes without negatively impacting program cost and schedule. Addressing producibility at this stage of the acquisition cycle is critical to achieving affordability objectives.","Additional Considerations: \n• Are the tooling and methods used applicable ultimately in production?\n• Have the manufacturing processes been assessed for capability to test and verify in production and their influence on Operations and Support (O&S)?\n• Have all relevant materials/processes been evaluated for manufacturability and producibility in a production relevant environment? \n• Can properly trained production personnel repeat the prototype work performed by engineers? \n• Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n• What are the ultimate manufacturing cost implications when in production?","Lessons Learned:\n• Historically manufacturability and producibility issues are ignored early in acquisition. \n• Producibility and manufacturability risks of the alternatives need to be presented to the decision makers before moving into EMD.\n• This is the opportune time to make design changes without negatively impacting program cost and schedule."]
[62,5,"B - Design","B.1 - Producibility Program","Have the manufacturing processes been assessed for their influence during the Operations and Support (O&S) phase?"]
[63,6,"B - Design","B.1 - Producibility Program","Have assessments and trade studies regarding performance vs. producibility for key technologies and/or components been completed?","Purpose: \nDemonstrate process capability in a production representative environment and to obtain fidelity in assessing producibility of the product by using data from building EMD units. Data generated from this assessment will be used to make design choices and do make-or-buy, capacity, process capability, sources, quality, yield/rate, and variability assessments. This information is used to determine what will be required to meet Production and Operations and Support requirements and identify needed producibility improvements. At this phase of a program the item/system must be manufactured in a production representative environment. Process capability and design producibility must be assessed against program requirements. It is essential to use actual manufacturing data while performing these assessments to truly identify producibility and manufacturing risk entering LRIP and subsequently transitioning to Full-Rate Production.","Additional Considerations: \n• Have the manufacturing processes been assessed for capability to test and verify in production and their influence on Operations and Support (O&S)?\n• Have all relevant materials/processes been evaluated for manufacturability and producibility in a production relevant environment? \n• Can properly trained production personnel repeat the prototype work performed by engineers? \n• Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n• What are the ultimate manufacturing cost implications when in production?","Lessons Learned: \n• It is crucial to continually address the producibility of hardware by assessing demonstrated process capability and, as more data becomes available, see if it will to meet production requirements."]
[64,6,"B - Design","B.1 - Producibility Program","Have the results of the producibility assessments and trade studies been used to shape the Acquisition Strategy, the Systems Engineering Plan, the plans for manufacturing and producibility plans, and planning for the Engineering and Manufacturing Development phase or for technology insertion programs?  ","Purpose: \nDemonstrate process capability in a production representative environment and to obtain fidelity in assessing producibility of the product by using data from building EMD units. Data generated from this assessment will be used to make design choices and do make-or-buy, capacity, process capability, sources, quality, yield/rate, and variability assessments. This information is used to determine what will be required to meet Production and Operations and Support requirements and identify needed producibility improvements. At this phase of a program the item/system must be manufactured in a production representative environment. Process capability and design producibility must be assessed against program requirements. It is essential to use actual manufacturing data while performing these assessments to truly identify producibility and manufacturing risk entering LRIP and subsequently transitioning to Full-Rate Production.","Additional Considerations: \n• Have the manufacturing processes been assessed for capability to test and verify in production and their influence on Operations and Support (O&S)?\n• Have all relevant materials/processes been evaluated for manufacturability and producibility in a production relevant environment? \n• Can properly trained production personnel repeat the prototype work performed by engineers? \n• Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n• What are the ultimate manufacturing cost implications when in production?","Lessons Learned: \n• It is crucial to continually address the producibility of hardware by assessing demonstrated process capability and, as more data becomes available, see if it will to meet production requirements."]
[65,6,"B - Design","B.1 - Producibility Program","Have the preliminary design choices been assessed against the constraints of the manufacturing processes and the capability of the industrial base?"]
[66,6,"B - Design","B.1 - Producibility Program","Have efforts to enhance the producibility (i.e., DFM, DFA, etc.) been initiated? ","Purpose: \nDemonstrate the producibility in a pilot line environment and obtain more fidelity assessing producibility/process capability by using data from building EMD units. To use this information to ensure that all producibility issues have been resolved, that LRIP requirements will be met, and to make producibility improvements on the system. At this phase of the system life-cycle the product should be built in a pilot line environment. The Data generated will be used to assess the producibility of the product against program requirements. It is essential to use actual manufacturing data while performing these assessments to truly identify producibility and manufacturing risk entering LRIP and subsequently transitioning to Full-Rate Production. \n\n","Additional Considerations: \n• Have the manufacturing processes been assessed for capability to test and verify in production and their influence on Operations and Support (O&S)?\n• Have all relevant materials/processes been evaluated for manufacturability and producibility in a production relevant environment? \n• Can properly trained production personnel repeat the prototype work performed by engineers? \n• Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n• What are the ultimate manufacturing cost implications when in production?","Lessons Learned: \n• It is important to continue to address the producibility of hardware by assessing demonstrated process capability and, as more data becomes available, see if it will meet production requirements."]
[67,7,"B - Design","B.1 - Producibility Program","Have detailed producibility trade studies using knowledge of key design characteristics and the related manufacturing process capability been completed?","Purpose: \nDemonstrate the producibility in a pilot line environment and obtain more fidelity assessing producibility/process capability by using data from building EMD units. To use this information to ensure that all producibility issues have been resolved, that LRIP requirements will be met, and to make producibility improvements on the system. At this phase of the system life-cycle the product should be built in a pilot line environment. The Data generated will be used to assess the producibility of the product against program requirements. It is essential to use actual manufacturing data while performing these assessments to truly identify producibility and manufacturing risk entering LRIP and subsequently transitioning to Full-Rate Production. \n\n","Additional Considerations: \n• Have the manufacturing processes been assessed for capability to test and verify in production and their influence on Operations and Support (O&S)?\n• Have all relevant materials/processes been evaluated for manufacturability and producibility in a production relevant environment? \n• Can properly trained production personnel repeat the prototype work performed by engineers? \n• Were manufacturing SMEs at least involved as observers of the prototypes being developed by systems/design engineering?\n• What are the ultimate manufacturing cost implications when in production?","Lessons Learned: \n• It is important to continue to address the producibility of hardware by assessing demonstrated process capability and, as more data becomes available, see if it will meet production requirements."]
[68,7,"B - Design","B.1 - Producibility Program","Are producibility enhancement efforts (i.e. Design for Manufacturing, Design for Assembly, etc.) ongoing for the optimized integrated system? ","Purpose: \nTo use demonstrated results from a LRIP line to verify producibility improvements have been effective and that processes are stable, adequately controlled, and capable to meet program objectives. Use the data to analyze the producibility to identify risk and areas for improvement. The program manager must constantly assess producibility by using demonstrated data from a LRIP environment to give confidence in meeting program objectives. Early production data will set the foundation for future production cost and performance. Management emphasis needs to be in the right place to ensure production cost and performance objectives can be achieved.","Additional Considerations: \n• What are the ultimate manufacturing cost implications when in production? \n• Are producibility and process capability assessments ongoing to identify possible cost reductions?","Lessons Learned: \n• Continually update assessments of product producibility as more data becomes available and use this data to assess whether program objectives can be met."]
[69,7,"B - Design","B.1 - Producibility Program","Have manufacturing processes been re-assessed as needed for capability to be tested and verified?","Purpose: \nTo use demonstrated results from a LRIP line to verify producibility improvements have been effective and that processes are stable, adequately controlled, and capable to meet program objectives. Use the data to analyze the producibility to identify risk and areas for improvement. The program manager must constantly assess producibility by using demonstrated data from a LRIP environment to give confidence in meeting program objectives. Early production data will set the foundation for future production cost and performance. Management emphasis needs to be in the right place to ensure production cost and performance objectives can be achieved.","Additional Considerations: \n• What are the ultimate manufacturing cost implications when in production? \n• Are producibility and process capability assessments ongoing to identify possible cost reductions?","Lessons Learned: \n• Continually update assessments of product producibility as more data becomes available and use this data to assess whether program objectives can be met."]
[70,7,"B - Design","B.1 - Producibility Program","Have manufacturing processes been re-assessed as needed for potential influence on the Operations and Support phase?","Purpose: \nDemonstrated producibility is continuing to be used to address all configuration changes to meet FRP objectives and improve cost. The program should maintain an effective producibility program addressing all configuration changes to ensure program objectives are being addressed.","Additional Considerations: \n• Are producibility and process capability assessments ongoing to identify possible cost reductions?","Lessons Learned: \n• Continually update assessments of product producibility as more data becomes available and use this data to assess whether program objectives can be met."]
[71,8,"B - Design","B.1 - Producibility Program","Have the producibility enhancement efforts (improvements) been implemented on system (for pilot line)?","Purpose: \nDemonstrated producibility is continuing to be used to address all configuration changes to meet FRP objectives and improve cost. The program should maintain an effective producibility program addressing all configuration changes to ensure program objectives are being addressed.","Additional Considerations: \n• Are producibility and process capability assessments ongoing to identify possible cost reductions?","Lessons Learned: \n• Continually update assessments of product producibility as more data becomes available and use this data to assess whether program objectives can be met."]
[72,8,"B - Design","B.1 - Producibility Program","Are all known producibility risks and issues being managed for Low Rate Initial Production (LRIP)?","Purpose: \nDemonstrated producibility is continuing to be used to address all configuration changes to meet FRP objectives and improve cost. The program should maintain an effective producibility program addressing all configuration changes to ensure program objectives are being addressed.","Additional Considerations: \n• Are producibility and process capability assessments ongoing to identify possible cost reductions?","Lessons Learned: \n• Continually update assessments of product producibility as more data becomes available and use this data to assess whether program objectives can be met."]
[73,9,"B - Design","B.1 - Producibility Program","Have prior producibility enhancement efforts (improvements) been analyzed for effectiveness during Low Rate Initial Production (LRIP)?","Purpose: \nThe focus at this lowest level of manufacturing readiness is to address manufacturing shortfalls and opportunities to advance the state of the art in manufacturing. Basic research (funded by budget) begins in the form of studies. MRL assessments should only be used here to obtain knowledge that could be useful to make informed leadership decisions on future manufacturing risks or technologies to address when proceeding into Applied Research. The purpose of manufacturing assessments in S&T is to transition capability to the warfighter more effectively and efficiently. Within Basic Research, assessments focus on identifying technology options that can be applied to the manufacturing process to support delivery of higher performance and more affordable Warfighter solutions.","Additional Considerations: \n• What are the major impediments to manufacturing with new materials or achieving streamlined manufacturing processes producing more affordable products?\n• What new Warfighter capabilities are being held back due to manufacturing costs or full scale productization solutions?\n• Where are we reaching the point of diminishing returns in manufacturing efficiency improvements due to limitations in current processes or availability of materials where alternate methods may offer new solutions?\n• Who are the major investment supporters for manufacturing S&T and what programs are they funding?\n• What are the new frontiers in manufacturing S&T where breakthroughs are likely to make a major impact?\n","Lessons Learned: \n• A community of interest approach to generating collaboration between S&T, requirements, acquisition and manufacturing is essential in communicating manufacturing challenges for future exploration by the research community."]
[74,9,"B - Design","B.1 - Producibility Program","Are any additional producibility issues and risks discovered in LRIP being managed for FRP?  ","Purpose: \nTo obtain knowledge that would be useful to leadership to make informed decisions on which future manufacturing risk areas or technologies they may wish to address when proceeding into the Applied Research phase or to define manufacturing areas where more basic research needs to be done. The purpose of these assessments is to help transition capability to the warfighter more effectively and efficiently. MRL use within Basic Research focuses on identifying technology options that can be applied to the manufacturing process to support delivery of higher performance and more affordable warfighter solutions.","Additional Considerations: \n• What are the major impediments to manufacturing with new materials or achieving streamlined manufacturing processes producing more affordable products?\n• What new Warfighter capabilities are being held back due to manufacturing costs or full scale productization solutions?\n• Where are we reaching the point of diminishing returns in manufacturing efficiency improvements due to limitations in current processes or availability of materials where alternate methods may offer new solutions?\n• Who are the major investment supporters for manufacturing S&T and what programs are they funding?\n• What are the new frontiers in manufacturing S&T where breakthroughs are likely to make a major impact?\n• What new manufacturing concepts are being explored and how do they apply to DOD manufacturing obstacles?\n","Lessons Learned: \n• A community of interest approach to generating collaboration between S&T, requirements, acquisition and manufacturing is essential in communicating manufacturing challenges for future exploration by the research community."]
[75,10,"B - Design","B.1 - Producibility Program","Have design producibility improvements been demonstrated in Full Rate Production (FRP)?","Purpose: \nTo obtain knowledge that would be useful to leadership to make informed decisions on which future manufacturing risk areas or technologies they may wish to address when proceeding into the Applied Research phase or to define manufacturing areas where more basic research needs to be done. The purpose of these assessments is to help transition capability to the warfighter more effectively and efficiently. MRL use within Basic Research focuses on identifying technology options that can be applied to the manufacturing process to support delivery of higher performance and more affordable warfighter solutions.","Additional Considerations: \n• What are the major impediments to manufacturing with new materials or achieving streamlined manufacturing processes producing more affordable products?\n• What new Warfighter capabilities are being held back due to manufacturing costs or full scale productization solutions?\n• Where are we reaching the point of diminishing returns in manufacturing efficiency improvements due to limitations in current processes or availability of materials where alternate methods may offer new solutions?\n• Who are the major investment supporters for manufacturing S&T and what programs are they funding?\n• What are the new frontiers in manufacturing S&T where breakthroughs are likely to make a major impact?\n• What new manufacturing concepts are being explored and how do they apply to DOD manufacturing obstacles?\n","Lessons Learned: \n• A community of interest approach to generating collaboration between S&T, requirements, acquisition and manufacturing is essential in communicating manufacturing challenges for future exploration by the research community."]
[76,10,"B - Design","B.1 - Producibility Program","Are process producibility improvements (continuous improvement enhancement efforts) ongoing?","Purpose: \nTo obtain knowledge that would be useful to leadership to make informed decisions on which future manufacturing risks or technologies they may wish to address when proceeding into the Applied Research phase, or to define manufacturing areas where more basic research needs to be done. The purpose of manufacturing assessments during R&D is to help transition capability to the warfighter more effectively and efficiently. MRL use within Basic Research focuses on identifying technology options that can be applied to the manufacturing process to support delivery of higher performance and more affordable warfighter solutions.","Additional Considerations: \n• What are the envisioned manufacturing challenges for the potential technologic solutions being considered for this capability?\n• Are new manufacturing technologies potentially required to address these challenges?\n• Are there new manufacturing processes needed to address the anticipated production requirements?\n• Who are the major investment supporters for manufacturing S&T and what programs are they funding?\n• What are the new frontiers in manufacturing S&T where breakthroughs are likely to make a major impact?\n","Lessons Learned: \n• Identifying new material usage or manufacturing needs or techniques that extend beyond the current production boundaries early is essential to making smart design decisions and avoiding manufacturing risk.\n• Early investment in manufacturing technology or process development is essential to avoiding major cost, schedule and performance issues during follow on acquisition phases of the program."]
[77,10,"B - Design","B.1 - Producibility Program","Are all modifications, upgrades, Diminishing Manufacturing Sources & Material Shortages mitigation efforts and other changes being assessed for producibility?","Purpose: \nTo obtain knowledge that would be useful to leadership to make informed decisions on which future manufacturing risks or technologies they may wish to address when proceeding into the Applied Research phase, or to define manufacturing areas where more basic research needs to be done. The purpose of manufacturing assessments during R&D is to help transition capability to the warfighter more effectively and efficiently. MRL use within Basic Research focuses on identifying technology options that can be applied to the manufacturing process to support delivery of higher performance and more affordable warfighter solutions.","Additional Considerations: \n• What are the envisioned manufacturing challenges for the potential technologic solutions being considered for this capability?\n• Are new manufacturing technologies potentially required to address these challenges?\n• Are there new manufacturing processes needed to address the anticipated production requirements?\n• Who are the major investment supporters for manufacturing S&T and what programs are they funding?\n• What are the new frontiers in manufacturing S&T where breakthroughs are likely to make a major impact?\n","Lessons Learned: \n• Identifying new material usage or manufacturing needs or techniques that extend beyond the current production boundaries early is essential to making smart design decisions and avoiding manufacturing risk.\n• Early investment in manufacturing technology or process development is essential to avoiding major cost, schedule and performance issues during follow on acquisition phases of the program."]
[78,1,"B - Design","B.2 - Design Maturity","Have current capability deficiencies and gaps been identified?","Purpose: \nTo obtain knowledge that would be useful to leadership to make informed decisions on which future manufacturing risks or technologies they may wish to address when proceeding into the Applied Research phase, or to define manufacturing areas where more basic research needs to be done. The purpose of manufacturing assessments during R&D is to help transition capability to the warfighter more effectively and efficiently. MRL use within Basic Research focuses on identifying technology options that can be applied to the manufacturing process to support delivery of higher performance and more affordable warfighter solutions.","Additional Considerations: \n• What are the envisioned manufacturing challenges for the potential technologic solutions being considered for this capability?\n• Are new manufacturing technologies potentially required to address these challenges?\n• Are there new manufacturing processes needed to address the anticipated production requirements?\n• Who are the major investment supporters for manufacturing S&T and what programs are they funding?\n• What are the new frontiers in manufacturing S&T where breakthroughs are likely to make a major impact?\n","Lessons Learned: \n• Identifying new material usage or manufacturing needs or techniques that extend beyond the current production boundaries early is essential to making smart design decisions and avoiding manufacturing risk.\n• Early investment in manufacturing technology or process development is essential to avoiding major cost, schedule and performance issues during follow on acquisition phases of the program."]
[79,2,"B - Design","B.2 - Design Maturity","Have analyses been performed to evaluate the feasibility of potential solutions to address capability gaps?","Purpose: \nTo ensure manufacturability, producibility, and quality processes are in place and sufficient to produce technology demonstrators. To identify manufacturing risks for building prototypes and ensure mitigation plans are in place. To identify manufacturing cost drivers and establish target cost objectives. Initial producibility assessments of design concepts have been completed. Key design performance parameters have been identified as well as special tooling, facilities, material handling and skills required. A manufacturing feasibility assessment at his time helps a program to: (1) Better understand the risk; (2) begin risk mitigation efforts, and (3) provides critical information to accurately reflect the financial risk in both estimating and funding processes. It is important before selecting any potential solution that manufacturing feasibility/ readiness be evaluated to understand the risk of achieving the cost and schedule for any proposed approach.","Additional Considerations:\n• Is the item design novel, or does it contain nonstandard dimensions or tolerances or arrangements?\n• Are there manufacturing technologies or processes not currently available that will lead to risk in advanced technology development?\n• Does DOD need to make investments to create new industrial capabilities to support a robust design?\n• What is the risk of industry not being able to provide new program design at planned cost and schedule?","Lessons Learned: \n• Identification of critical manufacturing technology and processes early on is critical to making low risk design trade-offs. \n• Of there are major manufacturing obstacles due to unavailable or immature manufacturing technology, alternative design choices should be considered."]
[80,3,"B - Design","B.2 - Design Maturity","Have high level performance, lifecycle, and technical requirements been defined?"]
[81,3,"B - Design","B.2 - Design Maturity","Have high level performance, lifecycle, and technical requirements been evaluated for system concepts?","Purpose: \nTo ensure manufacturability, producibility, and quality processes are in place and sufficient to produce technology demonstrators. To identify manufacturing risks for building prototypes and ensure mitigation plans are in place. To identify manufacturing cost drivers and establish target cost objectives. Initial producibility assessments of design concepts have been completed. Key design performance parameters have been identified as well as special tooling, facilities, material handling and skills required. A manufacturing feasibility assessment at his time helps a program to: (1) Better understand the risk; (2) begin risk mitigation efforts, and (3) provides critical information to accurately reflect the financial risk in both estimating and funding processes. It is important before selecting any potential solution that manufacturing feasibility/ readiness be evaluated to understand the risk of achieving the cost and schedule for any proposed approach.","Additional Considerations:\n• Is the item design novel, or does it contain nonstandard dimensions or tolerances or arrangements?\n• Are there manufacturing technologies or processes not currently available that will lead to risk in advanced technology development?\n• Does DOD need to make investments to create new industrial capabilities to support a robust design?\n• What is the risk of industry not being able to provide new program design at planned cost and schedule?","Lessons Learned: \n• Identification of critical manufacturing technology and processes early on is critical to making low risk design trade-offs. \n• Of there are major manufacturing obstacles due to unavailable or immature manufacturing technology, alternative design choices should be considered."]
[82,3,"B - Design","B.2 - Design Maturity","Are trade-offs in design options based on experiments and initial MOEs?","Purpose: \nTo ensure manufacturability, producibility, and quality processes are in place and sufficient to produce technology demonstrators. To identify manufacturing risks for building prototypes and ensure mitigation plans are in place. To identify manufacturing cost drivers and establish target cost objectives. Initial producibility assessments of design concepts have been completed. Key design performance parameters have been identified as well as special tooling, facilities, material handling and skills required. A manufacturing feasibility assessment at his time helps a program to: (1) Better understand the risk; (2) begin risk mitigation efforts, and (3) provides critical information to accurately reflect the financial risk in both estimating and funding processes. It is important before selecting any potential solution that manufacturing feasibility/ readiness be evaluated to understand the risk of achieving the cost and schedule for any proposed approach.","Additional Considerations:\n• Is the item design novel, or does it contain nonstandard dimensions or tolerances or arrangements?\n• Are there manufacturing technologies or processes not currently available that will lead to risk in advanced technology development?\n• Does DOD need to make investments to create new industrial capabilities to support a robust design?\n• What is the risk of industry not being able to provide new program design at planned cost and schedule?","Lessons Learned: \n• Identification of critical manufacturing technology and processes early on is critical to making low risk design trade-offs. \n• Of there are major manufacturing obstacles due to unavailable or immature manufacturing technology, alternative design choices should be considered."]
[83,4,"B - Design","B.2 - Design Maturity","Have form, fit, and function constraints, and manufacturing capabilities, been identified for the preferred systems concept?","Purpose: \nTo ensure manufacturability, producibility, and quality processes are in place and sufficient to produce technology demonstrators. To identify manufacturing risks for building prototypes and ensure mitigation plans are in place. To identify manufacturing cost drivers and establish target cost objectives. Initial producibility assessments of design concepts have been completed. Key design performance parameters have been identified as well as special tooling, facilities, material handling and skills required. A manufacturing feasibility assessment at his time helps a program to: (1) Better understand the risk; (2) begin risk mitigation efforts, and (3) provides critical information to accurately reflect the financial risk in both estimating and funding processes. It is important before selecting any potential solution that manufacturing feasibility/ readiness be evaluated to understand the risk of achieving the cost and schedule for any proposed approach.","Additional Considerations:\n• Is the item design novel, or does it contain nonstandard dimensions or tolerances or arrangements?\n• Are there manufacturing technologies or processes not currently available that will lead to risk in advanced technology development?\n• Does DOD need to make investments to create new industrial capabilities to support a robust design?\n• What is the risk of industry not being able to provide new program design at planned cost and schedule?","Lessons Learned: \n• Identification of critical manufacturing technology and processes early on is critical to making low risk design trade-offs. \n• Of there are major manufacturing obstacles due to unavailable or immature manufacturing technology, alternative design choices should be considered."]
[84,4,"B - Design","B.2 - Design Maturity","Does the SEP and T&E Strategy recognize the need for the establishment/validation of manufacturing capability and management of manufacturing risk for the product lifecycle?  "]
[85,4,"B - Design","B.2 - Design Maturity","Have initial Key Performance Parameters (KPPs) been identified for the preferred systems concept?","Purpose: \nTo identify and assess the manufacturing risk of proposed alternatives, to begin maturing the manufacturing processes to reduce risk, and assess the degree to which the system is adequately demonstrated in a relevant environment. The manufacturing risk identified in this phase must be addressed in EMD through risk mitigation efforts. Risk mitigation cost must also be reflected in cost estimates. The first step is to require the program to generate the design data needed for a manufacturing risk assessment and to introduce design related activities to mature manufacturing processes. The assessment process will define critical design activities needed to reach a target level of manufacturing maturity.","Additional Considerations: \n• Are there materials which have not been demonstrated in similar products or manufacturing processes? \n• Is this item a driver that significantly impacts life-cycle cost (development, unit, or operations and/or support costs)? Is the technology new with high cost uncertainty? \n• Is the item design novel or does it contain nonstandard dimensions, tolerances or arrangements? \n• Will the item require the use of manufacturing technology, processes, inspection, or capabilities that are unproven in the current environment? \n• Does the item have historical/anticipated yield or quality issues? \n• Are all required manufacturing processes and techniques currently available or are some still to be defined?\n• What is the probability of meeting the delivery date (e.g., for prototypes)?\n• What are the design producibility risks?\n• What is the potential impact of critical and long-lead time material?\n• Are there manufacturing capability, and cost/schedule impact analyses to support trade-offs among alternatives?\n","Lessons Learned: \n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[86,4,"B - Design","B.2 - Design Maturity","Have system characteristics and measures to support required capabilities been identified?","Purpose: \nTo identify and assess the manufacturing risk of proposed alternatives, to begin maturing the manufacturing processes to reduce risk, and assess the degree to which the system is adequately demonstrated in a relevant environment. The manufacturing risk identified in this phase must be addressed in EMD through risk mitigation efforts. Risk mitigation cost must also be reflected in cost estimates. The first step is to require the program to generate the design data needed for a manufacturing risk assessment and to introduce design related activities to mature manufacturing processes. The assessment process will define critical design activities needed to reach a target level of manufacturing maturity.","Additional Considerations: \n• Are there materials which have not been demonstrated in similar products or manufacturing processes? \n• Is this item a driver that significantly impacts life-cycle cost (development, unit, or operations and/or support costs)? Is the technology new with high cost uncertainty? \n• Is the item design novel or does it contain nonstandard dimensions, tolerances or arrangements? \n• Will the item require the use of manufacturing technology, processes, inspection, or capabilities that are unproven in the current environment? \n• Does the item have historical/anticipated yield or quality issues? \n• Are all required manufacturing processes and techniques currently available or are some still to be defined?\n• What is the probability of meeting the delivery date (e.g., for prototypes)?\n• What are the design producibility risks?\n• What is the potential impact of critical and long-lead time material?\n• Are there manufacturing capability, and cost/schedule impact analyses to support trade-offs among alternatives?\n","Lessons Learned: \n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[87,5,"B - Design","B.2 - Design Maturity","Have lower level performance requirements (e.g., those needed to meet initial KPPs) been analyzed as sufficient to proceed to the preliminary design?  ","Purpose: \nTo identify and assess the manufacturing risk of proposed alternatives, to begin maturing the manufacturing processes to reduce risk, and assess the degree to which the system is adequately demonstrated in a relevant environment. The manufacturing risk identified in this phase must be addressed in EMD through risk mitigation efforts. Risk mitigation cost must also be reflected in cost estimates. The first step is to require the program to generate the design data needed for a manufacturing risk assessment and to introduce design related activities to mature manufacturing processes. The assessment process will define critical design activities needed to reach a target level of manufacturing maturity.","Additional Considerations: \n• Are there materials which have not been demonstrated in similar products or manufacturing processes? \n• Is this item a driver that significantly impacts life-cycle cost (development, unit, or operations and/or support costs)? Is the technology new with high cost uncertainty? \n• Is the item design novel or does it contain nonstandard dimensions, tolerances or arrangements? \n• Will the item require the use of manufacturing technology, processes, inspection, or capabilities that are unproven in the current environment? \n• Does the item have historical/anticipated yield or quality issues? \n• Are all required manufacturing processes and techniques currently available or are some still to be defined?\n• What is the probability of meeting the delivery date (e.g., for prototypes)?\n• What are the design producibility risks?\n• What is the potential impact of critical and long-lead time material?\n• Are there manufacturing capability, and cost/schedule impact analyses to support trade-offs among alternatives?\n","Lessons Learned: \n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[88,5,"B - Design","B.2 - Design Maturity","Have all enabling and/or critical technologies and components (e.g., those needed to meet performance requirements) been identified? ","Purpose: \nTo identify and assess the manufacturing risk of proposed alternatives, to begin maturing the manufacturing processes to reduce risk, and assess the degree to which the system is adequately demonstrated in a relevant environment. The manufacturing risk identified in this phase must be addressed in EMD through risk mitigation efforts. Risk mitigation cost must also be reflected in cost estimates. The first step is to require the program to generate the design data needed for a manufacturing risk assessment and to introduce design related activities to mature manufacturing processes. The assessment process will define critical design activities needed to reach a target level of manufacturing maturity.","Additional Considerations: \n• Are there materials which have not been demonstrated in similar products or manufacturing processes? \n• Is this item a driver that significantly impacts life-cycle cost (development, unit, or operations and/or support costs)? Is the technology new with high cost uncertainty? \n• Is the item design novel or does it contain nonstandard dimensions, tolerances or arrangements? \n• Will the item require the use of manufacturing technology, processes, inspection, or capabilities that are unproven in the current environment? \n• Does the item have historical/anticipated yield or quality issues? \n• Are all required manufacturing processes and techniques currently available or are some still to be defined?\n• What is the probability of meeting the delivery date (e.g., for prototypes)?\n• What are the design producibility risks?\n• What is the potential impact of critical and long-lead time material?\n• Are there manufacturing capability, and cost/schedule impact analyses to support trade-offs among alternatives?\n","Lessons Learned: \n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[89,5,"B - Design","B.2 - Design Maturity","Has the product lifecycle been considered in identifying enabling and/or critical technologies and components?","Purpose: \nTo identify and assess the manufacturing risk of proposed alternatives, to begin maturing the manufacturing processes to reduce risk, and assess the degree to which the system is adequately demonstrated in a relevant environment. The manufacturing risk identified in this phase must be addressed in EMD through risk mitigation efforts. Risk mitigation cost must also be reflected in cost estimates. The first step is to require the program to generate the design data needed for a manufacturing risk assessment and to introduce design related activities to mature manufacturing processes. The assessment process will define critical design activities needed to reach a target level of manufacturing maturity.","Additional Considerations: \n• Are there materials which have not been demonstrated in similar products or manufacturing processes? \n• Is this item a driver that significantly impacts life-cycle cost (development, unit, or operations and/or support costs)? Is the technology new with high cost uncertainty? \n• Is the item design novel or does it contain nonstandard dimensions, tolerances or arrangements? \n• Will the item require the use of manufacturing technology, processes, inspection, or capabilities that are unproven in the current environment? \n• Does the item have historical/anticipated yield or quality issues? \n• Are all required manufacturing processes and techniques currently available or are some still to be defined?\n• What is the probability of meeting the delivery date (e.g., for prototypes)?\n• What are the design producibility risks?\n• What is the potential impact of critical and long-lead time material?\n• Are there manufacturing capability, and cost/schedule impact analyses to support trade-offs among alternatives?\n","Lessons Learned: \n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[90,5,"B - Design","B.2 - Design Maturity","Have the evaluation of the design for Key Characteristics (KCs) been initiated? ","Purpose: \nTo assess the manufacturing maturity of the program using MRL criteria to ensure recommended activities have been accomplished before proceeding into EMD, and to use design data created up to this point to assess the manufacturing risk and determine whether manufacturing processes will fall within the program’s cost and schedule objectives. To develop risk mitigation plans to minimize manufacturing risk in EMD. Demonstrating critical manufacturing processes in a production relevant environment is critical to assessing manufacturing risk and obtaining confidence you can achieve program cost, schedule and performance requirements before proceeding into EMD. To do this, the design must be matured for critical manufacturing technology areas. Once determined, pilot line production capability can be established to start manufacturing critical parts and components. The emphasis is on addressing higher risk areas (e.g., more advanced technologies and newer manufacturing capabilities). During this phase it is essential that the contractor demonstrate the capability to build the product, or a similar product, in the facility that will be used during production.","Additional Considerations:\n• Are there materials which have not been demonstrated in similar products or manufacturing processes? \n• Is this item a driver that significantly impacts life-cycle cost (development, unit, or operations and/or support costs)? Is the technology new with high cost uncertainty? \n• Is the item design novel or does it contain nonstandard dimensions, tolerances or arrangements? \n• Will the item require the use of manufacturing technology, processes, inspection, or capabilities that are unproven in the current environment? \n• Does the item have historical/anticipated yield or quality issues? \n• Are all required manufacturing processes and techniques currently available or are some still to be defined?\n• What is the probability of meeting the delivery date (e.g., for prototypes)?\n• What are the design producibility risks?\n• What is the potential impact of critical and long-lead time material?\n• Are there manufacturing capability, cost and schedule impact analyses to support trade-offs among alternatives?\n","Lessons Learned: \n• A production relevant environment is different for different products, which is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a production relevant environment for a satellite is different than that used in building prototype vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[91,5,"B - Design","B.2 - Design Maturity","Has the required product data to manufacture prototype components been released?","Purpose: \nTo assess the manufacturing maturity of the program using MRL criteria to ensure recommended activities have been accomplished before proceeding into EMD, and to use design data created up to this point to assess the manufacturing risk and determine whether manufacturing processes will fall within the program’s cost and schedule objectives. To develop risk mitigation plans to minimize manufacturing risk in EMD. Demonstrating critical manufacturing processes in a production relevant environment is critical to assessing manufacturing risk and obtaining confidence you can achieve program cost, schedule and performance requirements before proceeding into EMD. To do this, the design must be matured for critical manufacturing technology areas. Once determined, pilot line production capability can be established to start manufacturing critical parts and components. The emphasis is on addressing higher risk areas (e.g., more advanced technologies and newer manufacturing capabilities). During this phase it is essential that the contractor demonstrate the capability to build the product, or a similar product, in the facility that will be used during production.","Additional Considerations:\n• Are there materials which have not been demonstrated in similar products or manufacturing processes? \n• Is this item a driver that significantly impacts life-cycle cost (development, unit, or operations and/or support costs)? Is the technology new with high cost uncertainty? \n• Is the item design novel or does it contain nonstandard dimensions, tolerances or arrangements? \n• Will the item require the use of manufacturing technology, processes, inspection, or capabilities that are unproven in the current environment? \n• Does the item have historical/anticipated yield or quality issues? \n• Are all required manufacturing processes and techniques currently available or are some still to be defined?\n• What is the probability of meeting the delivery date (e.g., for prototypes)?\n• What are the design producibility risks?\n• What is the potential impact of critical and long-lead time material?\n• Are there manufacturing capability, cost and schedule impact analyses to support trade-offs among alternatives?\n","Lessons Learned: \n• A production relevant environment is different for different products, which is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a production relevant environment for a satellite is different than that used in building prototype vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[92,6,"B - Design","B.2 - Design Maturity","Has the allocated baseline for the system been established?","Purpose: \nTo assess the manufacturing maturity of the program using MRL criteria to ensure recommended activities have been accomplished before proceeding into EMD, and to use design data created up to this point to assess the manufacturing risk and determine whether manufacturing processes will fall within the program’s cost and schedule objectives. To develop risk mitigation plans to minimize manufacturing risk in EMD. Demonstrating critical manufacturing processes in a production relevant environment is critical to assessing manufacturing risk and obtaining confidence you can achieve program cost, schedule and performance requirements before proceeding into EMD. To do this, the design must be matured for critical manufacturing technology areas. Once determined, pilot line production capability can be established to start manufacturing critical parts and components. The emphasis is on addressing higher risk areas (e.g., more advanced technologies and newer manufacturing capabilities). During this phase it is essential that the contractor demonstrate the capability to build the product, or a similar product, in the facility that will be used during production.","Additional Considerations:\n• Are there materials which have not been demonstrated in similar products or manufacturing processes? \n• Is this item a driver that significantly impacts life-cycle cost (development, unit, or operations and/or support costs)? Is the technology new with high cost uncertainty? \n• Is the item design novel or does it contain nonstandard dimensions, tolerances or arrangements? \n• Will the item require the use of manufacturing technology, processes, inspection, or capabilities that are unproven in the current environment? \n• Does the item have historical/anticipated yield or quality issues? \n• Are all required manufacturing processes and techniques currently available or are some still to be defined?\n• What is the probability of meeting the delivery date (e.g., for prototypes)?\n• What are the design producibility risks?\n• What is the potential impact of critical and long-lead time material?\n• Are there manufacturing capability, cost and schedule impact analyses to support trade-offs among alternatives?\n","Lessons Learned: \n• A production relevant environment is different for different products, which is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a production relevant environment for a satellite is different than that used in building prototype vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[93,6,"B - Design","B.2 - Design Maturity","Are the product requirements and features well enough defined to support Preliminary Design Review (PDR)?","Purpose: \nTo assess the manufacturing maturity of the program using MRL criteria to ensure recommended activities have been accomplished before proceeding into EMD, and to use design data created up to this point to assess the manufacturing risk and determine whether manufacturing processes will fall within the program’s cost and schedule objectives. To develop risk mitigation plans to minimize manufacturing risk in EMD. Demonstrating critical manufacturing processes in a production relevant environment is critical to assessing manufacturing risk and obtaining confidence you can achieve program cost, schedule and performance requirements before proceeding into EMD. To do this, the design must be matured for critical manufacturing technology areas. Once determined, pilot line production capability can be established to start manufacturing critical parts and components. The emphasis is on addressing higher risk areas (e.g., more advanced technologies and newer manufacturing capabilities). During this phase it is essential that the contractor demonstrate the capability to build the product, or a similar product, in the facility that will be used during production.","Additional Considerations:\n• Are there materials which have not been demonstrated in similar products or manufacturing processes? \n• Is this item a driver that significantly impacts life-cycle cost (development, unit, or operations and/or support costs)? Is the technology new with high cost uncertainty? \n• Is the item design novel or does it contain nonstandard dimensions, tolerances or arrangements? \n• Will the item require the use of manufacturing technology, processes, inspection, or capabilities that are unproven in the current environment? \n• Does the item have historical/anticipated yield or quality issues? \n• Are all required manufacturing processes and techniques currently available or are some still to be defined?\n• What is the probability of meeting the delivery date (e.g., for prototypes)?\n• What are the design producibility risks?\n• What is the potential impact of critical and long-lead time material?\n• Are there manufacturing capability, cost and schedule impact analyses to support trade-offs among alternatives?\n","Lessons Learned: \n• A production relevant environment is different for different products, which is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a production relevant environment for a satellite is different than that used in building prototype vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[94,6,"B - Design","B.2 - Design Maturity","Has the required product data essential for manufacture of subsystem prototypes and/or the system prototypes been released?","Purpose: \nTo assess the manufacturing maturity of the program using MRL criteria to ensure recommended activities have been accomplished before proceeding into EMD, and to use design data created up to this point to assess the manufacturing risk and determine whether manufacturing processes will fall within the program’s cost and schedule objectives. To develop risk mitigation plans to minimize manufacturing risk in EMD. Demonstrating critical manufacturing processes in a production relevant environment is critical to assessing manufacturing risk and obtaining confidence you can achieve program cost, schedule and performance requirements before proceeding into EMD. To do this, the design must be matured for critical manufacturing technology areas. Once determined, pilot line production capability can be established to start manufacturing critical parts and components. The emphasis is on addressing higher risk areas (e.g., more advanced technologies and newer manufacturing capabilities). During this phase it is essential that the contractor demonstrate the capability to build the product, or a similar product, in the facility that will be used during production.","Additional Considerations:\n• Are there materials which have not been demonstrated in similar products or manufacturing processes? \n• Is this item a driver that significantly impacts life-cycle cost (development, unit, or operations and/or support costs)? Is the technology new with high cost uncertainty? \n• Is the item design novel or does it contain nonstandard dimensions, tolerances or arrangements? \n• Will the item require the use of manufacturing technology, processes, inspection, or capabilities that are unproven in the current environment? \n• Does the item have historical/anticipated yield or quality issues? \n• Are all required manufacturing processes and techniques currently available or are some still to be defined?\n• What is the probability of meeting the delivery date (e.g., for prototypes)?\n• What are the design producibility risks?\n• What is the potential impact of critical and long-lead time material?\n• Are there manufacturing capability, cost and schedule impact analyses to support trade-offs among alternatives?\n","Lessons Learned: \n• A production relevant environment is different for different products, which is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a production relevant environment for a satellite is different than that used in building prototype vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[95,6,"B - Design","B.2 - Design Maturity","Have all enabling and/or critical components been prototyped?"]
[96,6,"B - Design","B.2 - Design Maturity","Have all preliminary Key Characteristics (KCs) for the design been identified?","Purpose: \nTo reduce system level risk. EMD typically includes the demonstration of production prototype articles or engineering development models. EMD results in the completion of the design, or at least the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality. \n","Additional Considerations:\n• Is system design sufficient and stable enough to successfully pass CDR?\n• Are all technologies mature enough for production?\n• Are the design risks known and manageable?\n• Industrial base viability\n• Probability of meeting the delivery date (e.g., for qualification units)\n• Process maturity\n• Manufacturing costs\n• Supply chain management\n• Quality management\n• Facilities\n• Manufacturing skills availability","Lessons Learned: \n• A pilot line environment is different for different products, which is often dependent on rates and quantities of items being produced, and the complexity of those items. For example, a pilot line environment for a satellite is different than that used in building prototype vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[97,6,"B - Design","B.2 - Design Maturity","Have mitigation plans for Key Characteristics (KCs) been initiated?","Purpose: \nTo reduce system level risk. EMD typically includes the demonstration of production prototype articles or engineering development models. EMD results in the completion of the design, or at least the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality. \n","Additional Considerations:\n• Is system design sufficient and stable enough to successfully pass CDR?\n• Are all technologies mature enough for production?\n• Are the design risks known and manageable?\n• Industrial base viability\n• Probability of meeting the delivery date (e.g., for qualification units)\n• Process maturity\n• Manufacturing costs\n• Supply chain management\n• Quality management\n• Facilities\n• Manufacturing skills availability","Lessons Learned: \n• A pilot line environment is different for different products, which is often dependent on rates and quantities of items being produced, and the complexity of those items. For example, a pilot line environment for a satellite is different than that used in building prototype vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[98,7,"B - Design","B.2 - Design Maturity","Are the product design and features well enough defined to support Critical Design Review?","Purpose: \nTo reduce system level risk. EMD typically includes the demonstration of production prototype articles or engineering development models. EMD results in the completion of the design, or at least the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality. \n","Additional Considerations:\n• Is system design sufficient and stable enough to successfully pass CDR?\n• Are all technologies mature enough for production?\n• Are the design risks known and manageable?\n• Industrial base viability\n• Probability of meeting the delivery date (e.g., for qualification units)\n• Process maturity\n• Manufacturing costs\n• Supply chain management\n• Quality management\n• Facilities\n• Manufacturing skills availability","Lessons Learned: \n• A pilot line environment is different for different products, which is often dependent on rates and quantities of items being produced, and the complexity of those items. For example, a pilot line environment for a satellite is different than that used in building prototype vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[99,7,"B - Design","B.2 - Design Maturity","Has the impact of design change traffic on design maturity been evaluated as a risk or issue?","Purpose: \nTo reduce system level risk. EMD typically includes the demonstration of production prototype articles or engineering development models. EMD results in the completion of the design, or at least the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality. \n","Additional Considerations:\n• Is system design sufficient and stable enough to successfully pass CDR?\n• Are all technologies mature enough for production?\n• Are the design risks known and manageable?\n• Industrial base viability\n• Probability of meeting the delivery date (e.g., for qualification units)\n• Process maturity\n• Manufacturing costs\n• Supply chain management\n• Quality management\n• Facilities\n• Manufacturing skills availability","Lessons Learned: \n• A pilot line environment is different for different products, which is often dependent on rates and quantities of items being produced, and the complexity of those items. For example, a pilot line environment for a satellite is different than that used in building prototype vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[100,7,"B - Design","B.2 - Design Maturity","Has all of the product data essential for manufacture of products at the component level been released?"]
[101,7,"B - Design","B.2 - Design Maturity","Have risks and issues to Key Characteristics been identified?","Purpose: \nTo assure known producibility risks pose no significant challenges for low rate production and that cost model and yield and rate analyses have been updated with pilot line results. The Engineering and Manufacturing Development (EMD) phase results in the completion of the design, at least the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality. The MRL 8 evaluation is used to support a Milestone C decision to begin Low Rate Initial Production (LRIP).\n","Additional Considerations:\n• Has the system product design baseline been established and documented to enable hardware fabrication and software coding to proceed with proper configuration management? \n• Are adequate processes and metrics in place for the program to succeed? \n• Are the design risks known and manageable? \n• Are all technologies mature enough for production? \n• Is the detailed design producible within the production budget? \n• Is detail design complete and stable enough to enter low rate production? \n• Have manufacturing processes been demonstrated and proven in a pilot line environment? \n• Have all producibility trade studies and risk assessments been completed? \n• Is the production cost model based upon the stable detailed design and been validated? \n• Is the production cost model based upon the stable detailed design and been validated?\n• Are the production facilities ready and required workers trained?\n• Is the program schedule executable (technical/cost risks)?\n• Is the program properly staffed?\n• Does the system or subsystem have Information Assurance (IA) Certification and Accreditation approval from the system DAA? Is a signed authorization memo available? If the answer is no, contact the program’s Information Assurance POC to discuss potential risks.","Lessons Learned: \n• A pilot line environment is different for different products, which is often dependent on rates and quantities of items being produced, and the complexity of those items. For example, a pilot line environment for a satellite is different than that used in building prototype vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area. \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality. Suppliers may not understand the application of their products into the overall system. The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
[102,7,"B - Design","B.2 - Design Maturity","Are identified Key Characteristic (KC) risk mitigation plans in place?","Purpose: \nTo assure known producibility risks pose no significant challenges for low rate production and that cost model and yield and rate analyses have been updated with pilot line results. The Engineering and Manufacturing Development (EMD) phase results in the completion of the design, at least the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality. The MRL 8 evaluation is used to support a Milestone C decision to begin Low Rate Initial Production (LRIP).\n","Additional Considerations:\n• Has the system product design baseline been established and documented to enable hardware fabrication and software coding to proceed with proper configuration management? \n• Are adequate processes and metrics in place for the program to succeed? \n• Are the design risks known and manageable? \n• Are all technologies mature enough for production? \n• Is the detailed design producible within the production budget? \n• Is detail design complete and stable enough to enter low rate production? \n• Have manufacturing processes been demonstrated and proven in a pilot line environment? \n• Have all producibility trade studies and risk assessments been completed? \n• Is the production cost model based upon the stable detailed design and been validated? \n• Is the production cost model based upon the stable detailed design and been validated?\n• Are the production facilities ready and required workers trained?\n• Is the program schedule executable (technical/cost risks)?\n• Is the program properly staffed?\n• Does the system or subsystem have Information Assurance (IA) Certification and Accreditation approval from the system DAA? Is a signed authorization memo available? If the answer is no, contact the program’s Information Assurance POC to discuss potential risks.","Lessons Learned: \n• A pilot line environment is different for different products, which is often dependent on rates and quantities of items being produced, and the complexity of those items. For example, a pilot line environment for a satellite is different than that used in building prototype vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area. \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality. Suppliers may not understand the application of their products into the overall system. The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
[103,8,"B - Design","B.2 - Design Maturity","Is the detailed design of product features and interfaces complete?","Purpose: \nTo assure known producibility risks pose no significant challenges for low rate production and that cost model and yield and rate analyses have been updated with pilot line results. The Engineering and Manufacturing Development (EMD) phase results in the completion of the design, at least the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality. The MRL 8 evaluation is used to support a Milestone C decision to begin Low Rate Initial Production (LRIP).\n","Additional Considerations:\n• Has the system product design baseline been established and documented to enable hardware fabrication and software coding to proceed with proper configuration management? \n• Are adequate processes and metrics in place for the program to succeed? \n• Are the design risks known and manageable? \n• Are all technologies mature enough for production? \n• Is the detailed design producible within the production budget? \n• Is detail design complete and stable enough to enter low rate production? \n• Have manufacturing processes been demonstrated and proven in a pilot line environment? \n• Have all producibility trade studies and risk assessments been completed? \n• Is the production cost model based upon the stable detailed design and been validated? \n• Is the production cost model based upon the stable detailed design and been validated?\n• Are the production facilities ready and required workers trained?\n• Is the program schedule executable (technical/cost risks)?\n• Is the program properly staffed?\n• Does the system or subsystem have Information Assurance (IA) Certification and Accreditation approval from the system DAA? Is a signed authorization memo available? If the answer is no, contact the program’s Information Assurance POC to discuss potential risks.","Lessons Learned: \n• A pilot line environment is different for different products, which is often dependent on rates and quantities of items being produced, and the complexity of those items. For example, a pilot line environment for a satellite is different than that used in building prototype vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area. \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality. Suppliers may not understand the application of their products into the overall system. The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
[104,8,"B - Design","B.2 - Design Maturity","Has all the product data essential for manufacture of the system been released?","Purpose: \nTo assure known producibility risks pose no significant challenges for low rate production and that cost model and yield and rate analyses have been updated with pilot line results. The Engineering and Manufacturing Development (EMD) phase results in the completion of the design, at least the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality. The MRL 8 evaluation is used to support a Milestone C decision to begin Low Rate Initial Production (LRIP).\n","Additional Considerations:\n• Has the system product design baseline been established and documented to enable hardware fabrication and software coding to proceed with proper configuration management? \n• Are adequate processes and metrics in place for the program to succeed? \n• Are the design risks known and manageable? \n• Are all technologies mature enough for production? \n• Is the detailed design producible within the production budget? \n• Is detail design complete and stable enough to enter low rate production? \n• Have manufacturing processes been demonstrated and proven in a pilot line environment? \n• Have all producibility trade studies and risk assessments been completed? \n• Is the production cost model based upon the stable detailed design and been validated? \n• Is the production cost model based upon the stable detailed design and been validated?\n• Are the production facilities ready and required workers trained?\n• Is the program schedule executable (technical/cost risks)?\n• Is the program properly staffed?\n• Does the system or subsystem have Information Assurance (IA) Certification and Accreditation approval from the system DAA? Is a signed authorization memo available? If the answer is no, contact the program’s Information Assurance POC to discuss potential risks.","Lessons Learned: \n• A pilot line environment is different for different products, which is often dependent on rates and quantities of items being produced, and the complexity of those items. For example, a pilot line environment for a satellite is different than that used in building prototype vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area. \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality. Suppliers may not understand the application of their products into the overall system. The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
[105,8,"B - Design","B.2 - Design Maturity","Does the design change traffic significantly impact Low Rate Initial Production (LRIP)?","Purpose: \nTo ensure manufacturing process capability in a low rate production environment is at an appropriate quality level to meet design key characteristic tolerances, that LRIP cost targets have been met, and learning curves have been analyzed with actual data. The purpose of the Production and Deployment Phase is to achieve an operational capability that satisfies mission needs. The cost model has been developed for FRP environment and reflects the impact of continuous improvement. Operational test and evaluation will determine the effectiveness and suitability of the system. The MDA will make the decision to commit the Department of Defense to production at Milestone C and documents the decision in an ADM. Milestone C authorizes entry into LRIP (for MDAPs and major systems), into production or procurement (for non-major systems that do not require LRIP).","Additional Considerations:\n• Are there unresolved design changes that need to be implemented, and if so, what are the impacts on manufacturing?\n• Are the manufacturing processes under control?\n• Can the system be reliably produced?\n• Are critical processes under control?\n• Have key issues been resolved with appropriate changes to the Product Baseline?\n• Have necessary statutory and regulatory provisions been met?\n• Have all risks been identified, including appropriate mitigation plans?","Lessons Learned: \n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[106,8,"B - Design","B.2 - Design Maturity","Based on pilot line demonstration and evaluation, are all Key Characteristics attainable?","Purpose: \nTo ensure manufacturing process capability in a low rate production environment is at an appropriate quality level to meet design key characteristic tolerances, that LRIP cost targets have been met, and learning curves have been analyzed with actual data. The purpose of the Production and Deployment Phase is to achieve an operational capability that satisfies mission needs. The cost model has been developed for FRP environment and reflects the impact of continuous improvement. Operational test and evaluation will determine the effectiveness and suitability of the system. The MDA will make the decision to commit the Department of Defense to production at Milestone C and documents the decision in an ADM. Milestone C authorizes entry into LRIP (for MDAPs and major systems), into production or procurement (for non-major systems that do not require LRIP).","Additional Considerations:\n• Are there unresolved design changes that need to be implemented, and if so, what are the impacts on manufacturing?\n• Are the manufacturing processes under control?\n• Can the system be reliably produced?\n• Are critical processes under control?\n• Have key issues been resolved with appropriate changes to the Product Baseline?\n• Have necessary statutory and regulatory provisions been met?\n• Have all risks been identified, including appropriate mitigation plans?","Lessons Learned: \n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[107,9,"B - Design","B.2 - Design Maturity","Are the major product design features and the configuration stable?","Purpose: \nTo ensure manufacturing process capability in a low rate production environment is at an appropriate quality level to meet design key characteristic tolerances, that LRIP cost targets have been met, and learning curves have been analyzed with actual data. The purpose of the Production and Deployment Phase is to achieve an operational capability that satisfies mission needs. The cost model has been developed for FRP environment and reflects the impact of continuous improvement. Operational test and evaluation will determine the effectiveness and suitability of the system. The MDA will make the decision to commit the Department of Defense to production at Milestone C and documents the decision in an ADM. Milestone C authorizes entry into LRIP (for MDAPs and major systems), into production or procurement (for non-major systems that do not require LRIP).","Additional Considerations:\n• Are there unresolved design changes that need to be implemented, and if so, what are the impacts on manufacturing?\n• Are the manufacturing processes under control?\n• Can the system be reliably produced?\n• Are critical processes under control?\n• Have key issues been resolved with appropriate changes to the Product Baseline?\n• Have necessary statutory and regulatory provisions been met?\n• Have all risks been identified, including appropriate mitigation plans?","Lessons Learned: \n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[108,9,"B - Design","B.2 - Design Maturity","Has operational testing of production products validated the system design?","Purpose: \nTo ensure manufacturing process capability in a low rate production environment is at an appropriate quality level to meet design key characteristic tolerances, that LRIP cost targets have been met, and learning curves have been analyzed with actual data. The purpose of the Production and Deployment Phase is to achieve an operational capability that satisfies mission needs. The cost model has been developed for FRP environment and reflects the impact of continuous improvement. Operational test and evaluation will determine the effectiveness and suitability of the system. The MDA will make the decision to commit the Department of Defense to production at Milestone C and documents the decision in an ADM. Milestone C authorizes entry into LRIP (for MDAPs and major systems), into production or procurement (for non-major systems that do not require LRIP).","Additional Considerations:\n• Are there unresolved design changes that need to be implemented, and if so, what are the impacts on manufacturing?\n• Are the manufacturing processes under control?\n• Can the system be reliably produced?\n• Are critical processes under control?\n• Have key issues been resolved with appropriate changes to the Product Baseline?\n• Have necessary statutory and regulatory provisions been met?\n• Have all risks been identified, including appropriate mitigation plans?","Lessons Learned: \n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[109,9,"B - Design","B.2 - Design Maturity","Has a Physical Configuration Audit (or equivalent) determined conformance of the as-built product to the documented design and verified the product baseline?","Purpose: \nTo ensure manufacturing process capability in a low rate production environment is at an appropriate quality level to meet design key characteristic tolerances, that LRIP cost targets have been met, and learning curves have been analyzed with actual data. The purpose of the Production and Deployment Phase is to achieve an operational capability that satisfies mission needs. The cost model has been developed for FRP environment and reflects the impact of continuous improvement. Operational test and evaluation will determine the effectiveness and suitability of the system. The MDA will make the decision to commit the Department of Defense to production at Milestone C and documents the decision in an ADM. Milestone C authorizes entry into LRIP (for MDAPs and major systems), into production or procurement (for non-major systems that do not require LRIP).","Additional Considerations:\n• Are there unresolved design changes that need to be implemented, and if so, what are the impacts on manufacturing?\n• Are the manufacturing processes under control?\n• Can the system be reliably produced?\n• Are critical processes under control?\n• Have key issues been resolved with appropriate changes to the Product Baseline?\n• Have necessary statutory and regulatory provisions been met?\n• Have all risks been identified, including appropriate mitigation plans?","Lessons Learned: \n• Statistical Process Control is not a requirement for all programs. Rates and Quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[110,9,"B - Design","B.2 - Design Maturity","Is the design change traffic limited?","Purpose: \nTo ensure all materials, tooling, inspection and test equipment, facilities and manpower are in place and have met full rate production requirements, that rate production unit costs meet goals, and funding is sufficient for production at required rates. Lean practices are well established and continuous process improvements are ongoing. The FRP decision requires that manufacturing risk is understood and that the manufacturing processes for the system be capable, in control, and affordable. Prior to the FRP decision, a manufacturing readiness assessment should be conducted to ensure any outstanding risks will not impact the programs ability to deliver FRP requirements.","Additional Considerations:\n• Are there unresolved design changes that need to be implemented, and if so, what are the impacts on manufacturing?\n• Are the manufacturing processes under control?\n• Can the system be reliably produced?\n• Are critical processes under control?\n• Have key issues been resolved with appropriate changes to the Product Baseline?\n• Have necessary statutory and regulatory provisions been met?\n• Have all risks been identified, including appropriate mitigation plans?","\nLessons Learned: \n• Assessments of manufacturing readiness may be used to capture manufacturing product documentation. \n• It is a best practice to incorporate the preservation of such manufacturing product technical data packages in the Data Management Strategy."]
[111,9,"B - Design","B.2 - Design Maturity","Are all Key Characteristics (KC) controlled in Low Rate Initial Production (LRIP) to production quality levels?","Purpose: \nTo ensure all materials, tooling, inspection and test equipment, facilities and manpower are in place and have met full rate production requirements, that rate production unit costs meet goals, and funding is sufficient for production at required rates. Lean practices are well established and continuous process improvements are ongoing. The FRP decision requires that manufacturing risk is understood and that the manufacturing processes for the system be capable, in control, and affordable. Prior to the FRP decision, a manufacturing readiness assessment should be conducted to ensure any outstanding risks will not impact the programs ability to deliver FRP requirements.","Additional Considerations:\n• Are there unresolved design changes that need to be implemented, and if so, what are the impacts on manufacturing?\n• Are the manufacturing processes under control?\n• Can the system be reliably produced?\n• Are critical processes under control?\n• Have key issues been resolved with appropriate changes to the Product Baseline?\n• Have necessary statutory and regulatory provisions been met?\n• Have all risks been identified, including appropriate mitigation plans?","\nLessons Learned: \n• Assessments of manufacturing readiness may be used to capture manufacturing product documentation. \n• It is a best practice to incorporate the preservation of such manufacturing product technical data packages in the Data Management Strategy."]
[112,10,"B - Design","B.2 - Design Maturity","Is the product design stable?"]
[113,10,"B - Design","B.2 - Design Maturity","Are design changes limited to continuous improvement or to obsolescence?","Purpose: \nTo identify approaches that will be used in later phases to assess cost and understand cost risk in pursuing the design. Provide decision makers insight into the cost risk and associated mitigation options.","Additional Considerations: \n• Will new materials or product development concepts most likely be required?\n• Is a new technology or process available to process the material?\n• Scrap, rework and repair from low-yield processes must be considered in the total product cost estimate.\n• Approaches could include goals on how cost models will be used in high level trades (architectural concept selection & optimization), system level design, and flow-down to components (e.g., QFD with cost model weights, Pugh Concept Selection (PCS), CAIV, etc.).\n• Approaches could include how the life-cycle cost of the product will modeled including goal-setting (parameters could include R&D, transition, ramp/learning, sustainment, hidden factory, logistics, supplier, after-market, repair and disposal costs).\n","Lessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must start addressing what model we will use to assess the manufacturing cost earlier in the acquisition process. \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• In this early development phase it is extremely difficult to gain real manufacturing expertise experience to understand potentially critical manufacturing cost issues unless reviewers have had significant experience with Integrated Product and Process Development versus “silo” product development. \n• Cost estimates at this stage of the product life-cycle should be considered as ROM estimates and be open to adjustment as the design is firmed up."]
[114,10,"B - Design","B.2 - Design Maturity","Are all Key Characteristics (KCs) controlled in Full Rate Production (FRP) to planned quality levels?","Purpose: \nTo identify approaches that will be used in later phases to assess cost and understand cost risk in pursuing the design. Provide decision makers insight into the cost risk and associated mitigation options. To begin identifying cost targets and risk for the alternative being considered, and use this information to understand the cost risk of managing the transition. Put manufacturing cost on an equal footing with technical requirements. The project must project production cost by setting initial cost targets and identifying cost risks of producing alternative designs. At the end of Pre-MSA the best available alternatives will be selected to address entrance criteria for MSA.","Additional Considerations: \n• Affordability must be considered even during the early stages of a product life-cycle. Design and process engineers should be aware that designing a product cannot be done in a vacuum.\n• Consider a cost step-down approach, leveraging new/emerging technologies and manufacturing processes, and consider automation.\n• Production cost estimation can be by a number of means including formal Production Cost Models, parametric estimates, production actuals or cost risk bands based on technical maturity.\n• Cost targets can come from should-cost models which use existing internal products, future state products, competitor/best-in-class products, and customer price models.\n• Value stream maps showing high level architecture breakdown with suppliers aligned could be used to show goals for broken out costs.\n• Risks include large cost items, items with large variations, and new items with unknown costs.\n• Cost drivers might be material, labor, overhead, NRE, tooling, yields, rework, repair, etc. for any of the parts of the product life-cycle cost.\n","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• Cost estimates at this stage of the product life-cycle should be considered as ROM estimates and be open to adjustment as the design is firmed up."]
[115,1,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Have hypotheses been developed regarding technology impact on affordability?","Purpose: \nTo identify approaches that will be used in later phases to assess cost and understand cost risk in pursuing the design. Provide decision makers insight into the cost risk and associated mitigation options. To begin identifying cost targets and risk for the alternative being considered, and use this information to understand the cost risk of managing the transition. Put manufacturing cost on an equal footing with technical requirements. The project must project production cost by setting initial cost targets and identifying cost risks of producing alternative designs. At the end of Pre-MSA the best available alternatives will be selected to address entrance criteria for MSA.","Additional Considerations: \n• Affordability must be considered even during the early stages of a product life-cycle. Design and process engineers should be aware that designing a product cannot be done in a vacuum.\n• Consider a cost step-down approach, leveraging new/emerging technologies and manufacturing processes, and consider automation.\n• Production cost estimation can be by a number of means including formal Production Cost Models, parametric estimates, production actuals or cost risk bands based on technical maturity.\n• Cost targets can come from should-cost models which use existing internal products, future state products, competitor/best-in-class products, and customer price models.\n• Value stream maps showing high level architecture breakdown with suppliers aligned could be used to show goals for broken out costs.\n• Risks include large cost items, items with large variations, and new items with unknown costs.\n• Cost drivers might be material, labor, overhead, NRE, tooling, yields, rework, repair, etc. for any of the parts of the product life-cycle cost.\n","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• Cost estimates at this stage of the product life-cycle should be considered as ROM estimates and be open to adjustment as the design is firmed up."]
[116,2,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Has the cost model approach been defined?","Purpose: \nTo identify approaches that will be used in later phases to assess cost and understand cost risk in pursuing the design. Provide decision makers insight into the cost risk and associated mitigation options. To begin identifying cost targets and risk for the alternative being considered, and use this information to understand the cost risk of managing the transition. Put manufacturing cost on an equal footing with technical requirements. The project must project production cost by setting initial cost targets and identifying cost risks of producing alternative designs. At the end of Pre-MSA the best available alternatives will be selected to address entrance criteria for MSA.","Additional Considerations: \n• Affordability must be considered even during the early stages of a product life-cycle. Design and process engineers should be aware that designing a product cannot be done in a vacuum.\n• Consider a cost step-down approach, leveraging new/emerging technologies and manufacturing processes, and consider automation.\n• Production cost estimation can be by a number of means including formal Production Cost Models, parametric estimates, production actuals or cost risk bands based on technical maturity.\n• Cost targets can come from should-cost models which use existing internal products, future state products, competitor/best-in-class products, and customer price models.\n• Value stream maps showing high level architecture breakdown with suppliers aligned could be used to show goals for broken out costs.\n• Risks include large cost items, items with large variations, and new items with unknown costs.\n• Cost drivers might be material, labor, overhead, NRE, tooling, yields, rework, repair, etc. for any of the parts of the product life-cycle cost.\n","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• Cost estimates at this stage of the product life-cycle should be considered as ROM estimates and be open to adjustment as the design is firmed up."]
[117,3,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Has manufacturing cost estimates for system concepts been developed?","Purpose: \nTo break down production cost by identifying drivers in manufacturing, material, and special requirements for alternative designs, and to quantify the uncertainty of cost targets for a list of given assumptions (rate, process, material, production personnel, supplier capability, etc.). To use this information to understand the cost risk in selecting the best alternative. To ensure the validity behind cost targets by reviewing detail process charts and assumptions behind these targets and to quantify the uncertainty of the targets. The program must project production cost by setting cost targets and identifying cost risks to produce the selected alternative. At the end of MSA, the best available alternatives will be selected to address entrance criteria for TD, and you will understand the cost risk this decision presents in achieving program requirements\n","Additional Considerations: \n• These are budgetary estimates for design trades and should include BOE considerations.\n• Probable controlling influences on cost include materiel, processes, ITAR, capital investment, software, test, personnel, etc.\n• Cost drivers should be considered from all parts of the product life-cycle (e.g., R&D, transition to production, production ramp-up/learning, sustainment, hidden factory, logistics, repair and aftermarket, and disposal costs).\n• Cost targets can come from should-cost models which use similar existing internal products, supply chain competitor products, future state products, customer price models, etc.\n","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle. \n• Quantities and rates at this phase are notional."]
[118,3,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Have initial cost models been developed which include high-level process steps and materials? ","Purpose: \nTo break down production cost by identifying drivers in manufacturing, material, and special requirements for alternative designs, and to quantify the uncertainty of cost targets for a list of given assumptions (rate, process, material, production personnel, supplier capability, etc.). To use this information to understand the cost risk in selecting the best alternative. To ensure the validity behind cost targets by reviewing detail process charts and assumptions behind these targets and to quantify the uncertainty of the targets. The program must project production cost by setting cost targets and identifying cost risks to produce the selected alternative. At the end of MSA, the best available alternatives will be selected to address entrance criteria for TD, and you will understand the cost risk this decision presents in achieving program requirements\n","Additional Considerations: \n• These are budgetary estimates for design trades and should include BOE considerations.\n• Probable controlling influences on cost include materiel, processes, ITAR, capital investment, software, test, personnel, etc.\n• Cost drivers should be considered from all parts of the product life-cycle (e.g., R&D, transition to production, production ramp-up/learning, sustainment, hidden factory, logistics, repair and aftermarket, and disposal costs).\n• Cost targets can come from should-cost models which use similar existing internal products, supply chain competitor products, future state products, customer price models, etc.\n","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle. \n• Quantities and rates at this phase are notional."]
[120,4,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Have cost estimates been refined based on anticipated production volumes associated with preferred materiel solution?    ","Purpose: \nTo break down production cost by identifying drivers in manufacturing, material, and special requirements for alternative designs, and to quantify the uncertainty of cost targets for a list of given assumptions (rate, process, material, production personnel, supplier capability, etc.). To use this information to understand the cost risk in selecting the best alternative. To ensure the validity behind cost targets by reviewing detail process charts and assumptions behind these targets and to quantify the uncertainty of the targets. The program must project production cost by setting cost targets and identifying cost risks to produce the selected alternative. At the end of MSA, the best available alternatives will be selected to address entrance criteria for TD, and you will understand the cost risk this decision presents in achieving program requirements\n","Additional Considerations: \n• These are budgetary estimates for design trades and should include BOE considerations.\n• Probable controlling influences on cost include materiel, processes, ITAR, capital investment, software, test, personnel, etc.\n• Cost drivers should be considered from all parts of the product life-cycle (e.g., R&D, transition to production, production ramp-up/learning, sustainment, hidden factory, logistics, repair and aftermarket, and disposal costs).\n• Cost targets can come from should-cost models which use similar existing internal products, supply chain competitor products, future state products, customer price models, etc.\n","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle. \n• Quantities and rates at this phase are notional."]
[121,4,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Has cost model been updated with identified cost drivers (i.e., process variables, manufacturing, materials, and special requirements)?","Purpose: \nTo obtain more fidelity in the production cost estimates by using actual data from building prototypes or by using acceptable simulation models to focus on cost drivers. At this stage, details in the cost model should address specific manufacturing areas (e.g., materials, labor, equipment, tooling/Special Test Equipment, setup, yield/scrap/rework, Work In Progress, and capability/capacity constraints, etc.) for each prototype being built. The program manager must have confidence in the production cost to adequately assess the affordability of his final decision.","Additional Considerations: \n• There should be a reduced reliance on laboratory processes and resources at this stage.\n• Are design-to-cost (DTC) goals being met? \n• Is actual cost data from prototype builds being used?\n• Include quantities in the DTC estimate. DTC numbers should have more fidelity, if not completely firm, and assumptions should be clearly identified. \n• Are cost targets in place and cost drivers clearly understood?\n• If equipment, tooling, or STE are capitalized by the supplier, it may not be applicable to the analysis. Some level of clarification should be provided on when this data is considered.\n","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• Quantities and rates at this phase are still somewhat notional."]
[122,4,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Has the uncertainty of these cost drivers been quantified?","Purpose: \nTo obtain more fidelity in the production cost estimates by using actual data from building prototypes or by using acceptable simulation models to focus on cost drivers. At this stage, details in the cost model should address specific manufacturing areas (e.g., materials, labor, equipment, tooling/Special Test Equipment, setup, yield/scrap/rework, Work In Progress, and capability/capacity constraints, etc.) for each prototype being built. The program manager must have confidence in the production cost to adequately assess the affordability of his final decision.","Additional Considerations: \n• There should be a reduced reliance on laboratory processes and resources at this stage.\n• Are design-to-cost (DTC) goals being met? \n• Is actual cost data from prototype builds being used?\n• Include quantities in the DTC estimate. DTC numbers should have more fidelity, if not completely firm, and assumptions should be clearly identified. \n• Are cost targets in place and cost drivers clearly understood?\n• If equipment, tooling, or STE are capitalized by the supplier, it may not be applicable to the analysis. Some level of clarification should be provided on when this data is considered.\n","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• Quantities and rates at this phase are still somewhat notional."]
[123,4,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Does the cost model support AoA and ASR? "]
[124,5,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Are the end-to-end cost models driven by results of producing (or simulating production of) prototype components in a production relevant environment?","Purpose: \nTo obtain more fidelity in the production cost model it should be updated using data from building and testing the prototypes in a relevant environment. The program needs to have more realism in the production cost by using actual build data or acceptable simulation models to adequately assess affordability of final elected design. At Milestone B, the best available alternative will be selected to address user requirements, and the cost risk this decision presents will be understood.","Additional Considerations: \n• Include quantities in the DTC estimate. DTC numbers should have more fidelity, if not completely firm, and assumptions should be clearly identified. \n• Are cost targets in place and cost drivers clearly understood?\n","\nLessons Learned: \n• The cost model needs to be populated actual prototype build data as soon as possible, and it must continually be updated to reflect changes."]
[125,5,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Does the cost model include materials, labor, equipment, tooling, special test equipment, special inspection equipment, setup, yield, scrap, rework, Work in Progress, and capability and/or capacity constraints?"]
[126,6,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Is the cost model up-to-date?","Purpose: \nTo use demonstrated results from a production representative environment to verify production cost requirements can be met. The program must verify the realism in the cost model by using demonstrated data from a production representative environment to ensure confidence in affordability projections.","Additional Considerations: \n• Is actual cost data from prototype builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of LRIP and FRP manufacturing.","Lessons Learned: \n• The cost model needs to be populated with actual production representative build data, and it must continually be updated to reflect changes."]
[127,6,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","In addition to prior elements, does the updated cost model include design requirements, material specifications, tolerances, consideration of the Integrated Master Schedule, the results of system and/or subsystem simulations, and demonstrations of production relevant prototypes? ","Purpose: \nTo use demonstrated results from a pilot line environment to verify you can achieve you production cost requirements.","Additional Considerations: \n• None","Lessons Learned: \nThe key is to continually update your cost models as more data becomes available."]
[128,7,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Is the cost model up-to-date?","Purpose: \nTo use demonstrated results from a LRIP line to verify you can achieve you production rate cost requirements.","Additional Considerations: \n• None","Lessons Learned: \nThe key is to continually update your cost models as more data becomes available."]
[129,7,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","In addition to prior elements, does the updated cost model include the results of systems and/or sub-systems produced in a production representative environment, the production plant layout and design, and solutions to obsolescence risks and issues?","Purpose: \nMake our cost projections.","\nAdditional Considerations: \n• None","Lessons Learned: \nThe key is to continually update your cost models as more data becomes available."]
[130,8,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Is the cost model up-to-date with results of the pilot line build?","Purpose: \nTo begin identifying manufacturing cost implications, and to use this information to understand the risk in pursuing a given approach. To allow decision makers insight into the cost risk and start exploring mitigation options to address identified cost risks. Developers need to begin the process of identifying manufacturing cost implications for any approach being considered.","Additional Considerations: \n• What are the major impediments to manufacturing with new materials or achieving streamlined manufacturing processes producing more affordable products?\n• What new Warfighter capabilities are being held back due to manufacturing costs or full scale productization solutions?","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle."]
[131,9,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Has the Full Rate Production (FRP) cost model been updated with the result of the Low Rate Initial Production (LRIP) build?","\nPurpose: \nTo begin identifying manufacturing cost elements, and to use this information to understand the risk in pursuing a proposed approach. To allow decision makers insight into the cost risk and start exploring mitigation options to address identified cost risks. The S&T community needs to have a basic understanding on the manufacturing cost elements and implications of proceeding with the approach being considered.","Additional Considerations: \n• The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms.\n• Things to consider include yields/rates/volumes typical for the product line and/or market for which the technology is intended.\n","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle."]
[132,10,"C - Cost & Funding","C.1 - Production Cost Knowledge (Cost modeling)","Has the cost model been validated against actual Full Rate Production (FRP) cost?","Purpose: \nTo begin identifying the sensitivity of manufacturing cost drivers, and to use this information to understand the risk in pursuing this approach. To allow decision makers insight into the cost risk and start exploring mitigation options to address identified cost risks. To effectively transition ATD programs one must address manufacturing maturity of the prototypes being developed. Assessing the maturity of the manufacturing processes in ATDs mandates that the S&T and acquisition community work together. A key output of that assessment is a basic understanding of manufacturing cost drivers and their implications to the production development strategy.","Additional Considerations: \n• The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms.\n• Risks include large cost items, items with large variations, and new items with unknown costs.\n• Cost drivers might be material, labor, overhead, NRE, tooling, yields, rework, repair, etc. for any of the parts of the product life-cycle cost.","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle."]
[133,1,"C - Cost & Funding","C.2 - Cost Analysis","Have initial manufacturing and quality costs been identified?","Purpose: \nTo begin assessing the producibility of the alternatives being considered and ensure production cost models support the AoA. To use this information to understand the cost risk in selecting the alternatives to proceed with into the TD phase. To ensure the validity behind cost targets by reviewing detail process charts and assumptions behind these targets and to quantify the uncertainty of the targets. The program manager must begin projecting production cost and identifying cost risk to produce the alternative being considered. At the end of this phase the best available alternatives will be selected to proceed into TD. A producibility assessment should be performed to understand the cost risk presented by this decision.","Additional Considerations: \n• Cost analysis in the AoA can be done with a cost-weighted Pugh Concept Selection (PCS) or Quality Function Deployment (QFD) on those items with costs that can be modeled.\n","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• To accomplish this, use the data that is generated by a producibility analysis in the AoA and ASR."]
[134,2,"C - Cost & Funding","C.2 - Cost Analysis","Have potential manufacturing and quality cost drivers and system affordability gaps been identified?","Purpose: \nTo begin assessing the producibility of the alternatives being considered and ensure production cost models support the AoA. To use this information to understand the cost risk in selecting the alternatives to proceed with into the TD phase. To ensure the validity behind cost targets by reviewing detail process charts and assumptions behind these targets and to quantify the uncertainty of the targets. The program manager must begin projecting production cost and identifying cost risk to produce the alternative being considered. At the end of this phase the best available alternatives will be selected to proceed into TD. A producibility assessment should be performed to understand the cost risk presented by this decision.","Additional Considerations: \n• Cost analysis in the AoA can be done with a cost-weighted Pugh Concept Selection (PCS) or Quality Function Deployment (QFD) on those items with costs that can be modeled.\n","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• To accomplish this, use the data that is generated by a producibility analysis in the AoA and ASR."]
[135,3,"C - Cost & Funding","C.2 - Cost Analysis","Have analyses been conducted to refine manufacturing and quality cost drivers, risks, and development strategy (e.g., lab to pilot to factory)?"]
[136,3,"C - Cost & Funding","C.2 - Cost Analysis","Have potential cost reduction and system affordability gap closure strategies been identified?"]
[137,4,"C - Cost & Funding","C.2 - Cost Analysis","Have producibility and lifecycle cost risks and issues been assessed for the preferred materiel solution?  ","Purpose: \nTo obtain more fidelity in the production cost estimates by using data from building prototypes to assess whether target costs are achievable. To use data generated from cost analyses to make design choices, make/buy, capacity, process capability, sources, quality, key characteristics, yield/rate, and variability assessments. The program manager needs to obtain more realism assessing production cost by using actual prototype build data to ensure that his final decision will result in an affordable product. At the end of this phase, the best available alternatives will be selected to proceed into EMD. Any cost risk this decision presents in achieving program requirements will be understood.","Additional Considerations: \n• Are design-to-cost (DTC) goals being met? \n• Is actual cost data from prototype builds being used?\n• Include quantities in the DTC estimate. DTC numbers should have more fidelity, if not completely firm, and assumptions should be clearly identified. \n• Are cost targets in place and cost drivers clearly understood?\n• If equipment, tooling, or STE are capitalized by the supplier, it may not be applicable to the analysis. Some level of clarification should be provided on when this data is considered.\n","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• The need for realism of the cost model requires the use of actual build data as soon as possible. This data will impact the affordability of alternatives being considered."]
[138,4,"C - Cost & Funding","C.2 - Cost Analysis","Does the initial cost analysis support the Analysis of Alternatives (AoA)?","Purpose: \nTo obtain more fidelity in the production cost estimates by using data from building prototypes to assess whether target costs are achievable. To use data generated from cost analyses to make design choices, make/buy, capacity, process capability, sources, quality, key characteristics, yield/rate, and variability assessments. The program manager needs to obtain more realism assessing production cost by using actual prototype build data to ensure that his final decision will result in an affordable product. At the end of this phase, the best available alternatives will be selected to proceed into EMD. Any cost risk this decision presents in achieving program requirements will be understood.","Additional Considerations: \n• Are design-to-cost (DTC) goals being met? \n• Is actual cost data from prototype builds being used?\n• Include quantities in the DTC estimate. DTC numbers should have more fidelity, if not completely firm, and assumptions should be clearly identified. \n• Are cost targets in place and cost drivers clearly understood?\n• If equipment, tooling, or STE are capitalized by the supplier, it may not be applicable to the analysis. Some level of clarification should be provided on when this data is considered.\n","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• The need for realism of the cost model requires the use of actual build data as soon as possible. This data will impact the affordability of alternatives being considered."]
[139,4,"C - Cost & Funding","C.2 - Cost Analysis","Does the initial cost analysis support the Alternative Systems Review (ASR)?","Purpose: \nTo obtain more fidelity in the production cost model (i.e. at system/sub-system level) by updating it using data from building and testing prototypes in a relevant environment. To use this data to identify cost targets and drivers and begin efforts for cost reduction strategies. To provide this information for the “Should-Cost” models. The program manager needs to obtain more realism assessing production cost by using actual prototype build data to ensure that his final decision will result in an affordable product. At the end of this phase, the best available alternatives will be selected to proceed into EMD. Any cost risk this decision presents in achieving program requirements will be understood.","Additional Considerations: \n• You need actual build costs to perform an accurate Should-Cost review.","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• The need for realism of the cost model requires the use of actual build data as soon as possible. This data will impact the affordability of alternatives being considered."]
[141,5,"C - Cost & Funding","C.2 - Cost Analysis","Have prototype components' actual costs been included in cost analyses to validate achievability of target costs?","Purpose: \nTo obtain more fidelity in the production cost model (i.e. at system/sub-system level) by updating it using data from building and testing prototypes in a relevant environment. To use this data to identify cost targets and drivers and begin efforts for cost reduction strategies. To provide this information for the “Should-Cost” models. The program manager needs to obtain more realism assessing production cost by using actual prototype build data to ensure that his final decision will result in an affordable product. At the end of this phase, the best available alternatives will be selected to proceed into EMD. Any cost risk this decision presents in achieving program requirements will be understood.","Additional Considerations: \n• You need actual build costs to perform an accurate Should-Cost review.","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• The need for realism of the cost model requires the use of actual build data as soon as possible. This data will impact the affordability of alternatives being considered."]
[142,5,"C - Cost & Funding","C.2 - Cost Analysis","Is the up-to-date cost model being utilized as an input to decisions (trades) on design, make/buy choices, capacity requirements, process capability requirements, sources, quality requirements, Key Characteristics, yield/rate requirements, and variability control requirements?    ","Purpose: \nTo obtain more fidelity in the production cost model (i.e. at system/sub-system level) by updating it using data from building and testing prototypes in a relevant environment. To use this data to identify cost targets and drivers and begin efforts for cost reduction strategies. To provide this information for the “Should-Cost” models. The program manager needs to obtain more realism assessing production cost by using actual prototype build data to ensure that his final decision will result in an affordable product. At the end of this phase, the best available alternatives will be selected to proceed into EMD. Any cost risk this decision presents in achieving program requirements will be understood.","Additional Considerations: \n• You need actual build costs to perform an accurate Should-Cost review.","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• The need for realism of the cost model requires the use of actual build data as soon as possible. This data will impact the affordability of alternatives being considered."]
[143,6,"C - Cost & Funding","C.2 - Cost Analysis","Have prototype system and/or subsystems' actual costs been included in cost analyses to validate achievability of target costs? ","Purpose: \nTo obtain more fidelity in the production cost model (i.e. at system/sub-system level) by updating it using data from building and testing prototypes in a relevant environment. To use this data to identify cost targets and drivers and begin efforts for cost reduction strategies. To provide this information for the “Should-Cost” models. The program manager needs to obtain more realism assessing production cost by using actual prototype build data to ensure that his final decision will result in an affordable product. At the end of this phase, the best available alternatives will be selected to proceed into EMD. Any cost risk this decision presents in achieving program requirements will be understood.","Additional Considerations: \n• You need actual build costs to perform an accurate Should-Cost review.","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if identified and implemented early in the product life-cycle.\n• The need for realism of the cost model requires the use of actual build data as soon as possible. This data will impact the affordability of alternatives being considered."]
[144,6,"C - Cost & Funding","C.2 - Cost Analysis","Have cost targets been allocated to subsystems?","Purpose: \nTo use demonstrated results from a production representative environment to verify you can achieve production cost requirements. To use data for trade studies and engineering change and to start implementing your cost reduction strategies. To update the “Should-Cost” model with actuals from the production represented environment. The program needs to verify the realism of the cost model by using actual cost data from a production representative environment to ensure confidence in system affordability projections. The program needs to use the cost data to make design and manufacturing decisions, and ensure cost reduction activities have started.","Additional Considerations: \n• You need actual build costs to perform an accurate Should-Cost review.\n• Costs for subsystems and materials must be applicable to the objective military system design.","Lessons Learned: \n• The need for realism of the cost model requires the use of actual build data. This data will impact the affordability of alternatives being considered.\n• The cost model needs to be populated with actual production representative build data, and it must continually be updated to reflect changes.\n• A program must stay focused on the cost reduction effort at the beginning of EMD, or when you transition into production you will use the inefficiencies in the past to justify the cost for production versus what it should cost."]
[145,6,"C - Cost & Funding","C.2 - Cost Analysis","Have cost reduction and avoidance strategies been developed?","Purpose: \nTo use demonstrated results from a production representative environment to verify you can achieve production cost requirements. To use data for trade studies and engineering change and to start implementing your cost reduction strategies. To update the “Should-Cost” model with actuals from the production represented environment. The program needs to verify the realism of the cost model by using actual cost data from a production representative environment to ensure confidence in system affordability projections. The program needs to use the cost data to make design and manufacturing decisions, and ensure cost reduction activities have started.","Additional Considerations: \n• You need actual build costs to perform an accurate Should-Cost review.\n• Costs for subsystems and materials must be applicable to the objective military system design.","Lessons Learned: \n• The need for realism of the cost model requires the use of actual build data. This data will impact the affordability of alternatives being considered.\n• The cost model needs to be populated with actual production representative build data, and it must continually be updated to reflect changes.\n• A program must stay focused on the cost reduction effort at the beginning of EMD, or when you transition into production you will use the inefficiencies in the past to justify the cost for production versus what it should cost."]
[146,6,"C - Cost & Funding","C.2 - Cost Analysis","Have manufacturing cost drivers for the \"Should-Cost\" models been provided?","Purpose: \nTo use demonstrated results from a production representative environment to verify you can achieve production cost requirements. To use data for trade studies and engineering change and to start implementing your cost reduction strategies. To update the “Should-Cost” model with actuals from the production represented environment. The program needs to verify the realism of the cost model by using actual cost data from a production representative environment to ensure confidence in system affordability projections. The program needs to use the cost data to make design and manufacturing decisions, and ensure cost reduction activities have started.","Additional Considerations: \n• You need actual build costs to perform an accurate Should-Cost review.\n• Costs for subsystems and materials must be applicable to the objective military system design.","Lessons Learned: \n• The need for realism of the cost model requires the use of actual build data. This data will impact the affordability of alternatives being considered.\n• The cost model needs to be populated with actual production representative build data, and it must continually be updated to reflect changes.\n• A program must stay focused on the cost reduction effort at the beginning of EMD, or when you transition into production you will use the inefficiencies in the past to justify the cost for production versus what it should cost."]
[147,7,"C - Cost & Funding","C.2 - Cost Analysis","Have all available actual manufacturing costs been rolled up to the subsystem and/or system level?","Purpose: \nTo use demonstrated results from a production representative environment to verify you can achieve production cost requirements. To use data for trade studies and engineering change and to start implementing your cost reduction strategies. To update the “Should-Cost” model with actuals from the production represented environment. The program needs to verify the realism of the cost model by using actual cost data from a production representative environment to ensure confidence in system affordability projections. The program needs to use the cost data to make design and manufacturing decisions, and ensure cost reduction activities have started.","Additional Considerations: \n• You need actual build costs to perform an accurate Should-Cost review.\n• Costs for subsystems and materials must be applicable to the objective military system design.","Lessons Learned: \n• The need for realism of the cost model requires the use of actual build data. This data will impact the affordability of alternatives being considered.\n• The cost model needs to be populated with actual production representative build data, and it must continually be updated to reflect changes.\n• A program must stay focused on the cost reduction effort at the beginning of EMD, or when you transition into production you will use the inefficiencies in the past to justify the cost for production versus what it should cost."]
[148,7,"C - Cost & Funding","C.2 - Cost Analysis","Have all available actual manufacturing costs been tracked against targets?"]
[149,7,"C - Cost & Funding","C.2 - Cost Analysis","Is the up-to-date cost model and life cycle cost estimate being utilized as an input to decisions on detailed design trade studies and engineering change requests? ","Purpose: \nTo use demonstrated results from a production representative environment to verify you can achieve production cost requirements. To use data for trade studies and engineering change and to start implementing your cost reduction strategies. To update the “Should-Cost” model with actuals from the production represented environment. The program needs to verify the realism of the cost model by using actual cost data from a production representative environment to ensure confidence in system affordability projections. The program needs to use the cost data to make design and manufacturing decisions, and ensure cost reduction activities have started.","Additional Considerations: \n• Is actual cost data from pilot line builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of LRIP and FRP manufacturing.","Lessons Learned: \n• Though the cost model may be using actual pilot line build data, it must continually be updated to reflect changes.\n• When updating the cost model with pilot line build cost data, it may be appropriate to leverage similar technology/process data to “tune” the model (e.g., transition from Pilot Line to Production shows standard learning curves may not be appropriate)."]
[150,7,"C - Cost & Funding","C.2 - Cost Analysis","Are cost reduction and avoidance strategies being implemented (i.e., underway)?","Purpose: \nTo use demonstrated results from a production representative environment to verify you can achieve production cost requirements. To use data for trade studies and engineering change and to start implementing your cost reduction strategies. To update the “Should-Cost” model with actuals from the production represented environment. The program needs to verify the realism of the cost model by using actual cost data from a production representative environment to ensure confidence in system affordability projections. The program needs to use the cost data to make design and manufacturing decisions, and ensure cost reduction activities have started.","Additional Considerations: \n• Is actual cost data from pilot line builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of LRIP and FRP manufacturing.","Lessons Learned: \n• Though the cost model may be using actual pilot line build data, it must continually be updated to reflect changes.\n• When updating the cost model with pilot line build cost data, it may be appropriate to leverage similar technology/process data to “tune” the model (e.g., transition from Pilot Line to Production shows standard learning curves may not be appropriate)."]
[151,7,"C - Cost & Funding","C.2 - Cost Analysis","Have the manufacturing cost drivers for the \"Should-Cost\" models been updated?","Purpose: \nTo use demonstrated results from a production representative environment to verify you can achieve production cost requirements. To use data for trade studies and engineering change and to start implementing your cost reduction strategies. To update the “Should-Cost” model with actuals from the production represented environment. The program needs to verify the realism of the cost model by using actual cost data from a production representative environment to ensure confidence in system affordability projections. The program needs to use the cost data to make design and manufacturing decisions, and ensure cost reduction activities have started.","Additional Considerations: \n• Is actual cost data from pilot line builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of LRIP and FRP manufacturing.","Lessons Learned: \n• Though the cost model may be using actual pilot line build data, it must continually be updated to reflect changes.\n• When updating the cost model with pilot line build cost data, it may be appropriate to leverage similar technology/process data to “tune” the model (e.g., transition from Pilot Line to Production shows standard learning curves may not be appropriate)."]
[152,8,"C - Cost & Funding","C.2 - Cost Analysis","Have pilot line actual costs been included in cost analyses to verify achievability of target costs?"]
[153,8,"C - Cost & Funding","C.2 - Cost Analysis","Have proposed changes to requirements or configuration been subject to a manufacturing cost analysis?","Purpose: \nTo use demonstrated results from a LRIP line to verify you can achieve your production rate cost requirements. To use the data to analyze the labor efficiency, for Should-Cost estimates, and to identify areas for improvement. The program must constantly verify realism of the cost model by using actual build data from LRIP to ensure confidence in affordability projections. This early production data will set the foundation for future production cost, so you must understand where management emphasis needs to be placed to reduce cost and ensure production cost objectives can be achieved.","Additional Considerations: \n• Is actual cost data from LRIP builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of FRP manufacturing.","Lessons Learned: \n• The cost model must be using actual FRP build data, and it should continually be updated as more data becomes available.\n• Use this data to update the Should Cost estimate for the program."]
[154,8,"C - Cost & Funding","C.2 - Cost Analysis","Are the cost reduction and cost avoidance initiatives ongoing?"]
[155,8,"C - Cost & Funding","C.2 - Cost Analysis","Have the manufacturing cost drivers for the \"Should-Cost\" models been updated?","Purpose: \nTo use demonstrated results from a LRIP line to verify you can achieve your production rate cost requirements. To use the data to analyze the labor efficiency, for Should-Cost estimates, and to identify areas for improvement. The program must constantly verify realism of the cost model by using actual build data from LRIP to ensure confidence in affordability projections. This early production data will set the foundation for future production cost, so you must understand where management emphasis needs to be placed to reduce cost and ensure production cost objectives can be achieved.","Additional Considerations: \n• Is actual cost data from LRIP builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of FRP manufacturing.","Lessons Learned: \n• The cost model must be using actual FRP build data, and it should continually be updated as more data becomes available.\n• Use this data to update the Should Cost estimate for the program."]
[156,9,"C - Cost & Funding","C.2 - Cost Analysis","Have Low Rate Initial Production (LRIP) cost goals been met?","Purpose: \nTo use demonstrated results from a LRIP line to verify you can achieve your production rate cost requirements. To use the data to analyze the labor efficiency, for Should-Cost estimates, and to identify areas for improvement. The program must constantly verify realism of the cost model by using actual build data from LRIP to ensure confidence in affordability projections. This early production data will set the foundation for future production cost, so you must understand where management emphasis needs to be placed to reduce cost and ensure production cost objectives can be achieved.","Additional Considerations: \n• Is actual cost data from LRIP builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of FRP manufacturing.","Lessons Learned: \n• The cost model must be using actual FRP build data, and it should continually be updated as more data becomes available.\n• Use this data to update the Should Cost estimate for the program."]
[157,9,"C - Cost & Funding","C.2 - Cost Analysis","Have the learning curves been analyzed with actual data? ","Purpose: \nTo use demonstrated results from a LRIP line to verify you can achieve your production rate cost requirements. To use the data to analyze the labor efficiency, for Should-Cost estimates, and to identify areas for improvement. The program must constantly verify realism of the cost model by using actual build data from LRIP to ensure confidence in affordability projections. This early production data will set the foundation for future production cost, so you must understand where management emphasis needs to be placed to reduce cost and ensure production cost objectives can be achieved.","Additional Considerations: \n• Is actual cost data from LRIP builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of FRP manufacturing.","Lessons Learned: \n• The cost model must be using actual FRP build data, and it should continually be updated as more data becomes available.\n• Use this data to update the Should Cost estimate for the program."]
[158,9,"C - Cost & Funding","C.2 - Cost Analysis","Are the cost reduction and cost avoidance initiatives ongoing? ","Purpose: \nMake accurate cost projections. The program actual production build cost data should support affordability projections and provide cost-reduction opportunities.","Additional Considerations: \n• Is actual cost data from FRP builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of FRP manufacturing.","Lessons Learned: \n• The cost model should at this point look at obsolescence issues and product refresh. These could change the cost model but might not be considered.\n• If the design changes for any reason the system might be at a lower MRL after the design change. Many organizations will still consider this as Full Rate Production even though process technologies need to change."]
[159,9,"C - Cost & Funding","C.2 - Cost Analysis","Has touch labor efficiency been analyzed to meet production rates?","Purpose: \nMake accurate cost projections. The program actual production build cost data should support affordability projections and provide cost-reduction opportunities.","Additional Considerations: \n• Is actual cost data from FRP builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of FRP manufacturing.","Lessons Learned: \n• The cost model should at this point look at obsolescence issues and product refresh. These could change the cost model but might not be considered.\n• If the design changes for any reason the system might be at a lower MRL after the design change. Many organizations will still consider this as Full Rate Production even though process technologies need to change."]
[160,9,"C - Cost & Funding","C.2 - Cost Analysis","Have elements of inefficiency for touch labor been identified with plans in place for reduction?","Purpose: \nTo begin identify manufacturing cost implications, and to use this information to allow decision makers insight into potential investments to address identified cost risks. Researchers need to begin a process for understanding the manufacturing cost implications of their approaches and to consider investment options to reduce risk.","Additional Considerations: \n• What are the major impediments to manufacturing with new materials or achieving streamlined manufacturing processes producing more affordable products?","Lessons Learned: \n• The process of cost estimation and allocation has the highest probability of success if assessing cost and identifying potential investments to reduce cost starts before hardware development really begins."]
[161,10,"C - Cost & Funding","C.2 - Cost Analysis","Have Full Rate Production (FRP) cost goals been met?","Purpose: \nTo assess whether there are adequate funds budgeted to reach an MRL 3. The S&T community needs to have funds available on R&D programs to address manufacturing maturity, and if it is anticipated that there will be hardware migrating to the next phase, there should be a budget plan to achieve an MRL 3 before completing the R&D effort (i.e., Applied Research or Advanced Technology Development). \n","Additional Considerations: \n• The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms and manufacturing investments should be considered to address these issues.\n• Critical issues may involve large cost items, items with large variations, and new items with unknown costs.\n• Cost drivers might be material, labor, overhead, NRE, tooling, yields, rework, repair, etc. for any of the parts of the product life-cycle cost.\n","Lessons Learned: \n• Product affordability has the highest likelihood of being achieved if identification of investments to reduce cost is completed at during hardware development.\n• Budgeting efforts should be instituted to mature the manufacturing processes and to understand the risk."]
[162,10,"C - Cost & Funding","C.2 - Cost Analysis","Are the cost reduction and cost avoidance initiatives continuing?","Purpose: \nTo make sure that there is a reasonable budget to mature the manufacturing process to a MRL 4. To effectively transition S&T or R&D projects one must address the manufacturing maturity of the prototypes being developed. Budgeting the funds to reach a MRL 4 on these projects will require the S&T and acquisition community working together to ensure this is being done.","Additional Considerations: \n• The government program manager, the contractor, and the service-specific ManTech organization need to have a dialogue on critical issues that drive affordability, especially if the process has applicability to multiple platforms and manufacturing investments should be considered to address these issues.\n• Risks include large cost items, items with large variations, and new items with unknown costs.\n• Potential investments might be in material, labor, major NRE (internal and suppliers), tooling, facilities, etc.","Lessons Learned: \n• Product affordability has the highest likelihood of being achieved if potential investments to reduce cost are programmed during the process of hardware development.\n• Budgeting efforts should be instituted to mature the manufacturing processes and to understand the risk."]
[163,1,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Has a potential manufacturing investment strategy been developed?","Purpose: \nTo understand the budget available to mitigate the manufacturing risk of proceeding to Milestone A and ensure that a reasonable budget is in place to mature the manufacturing processes to an MRL 6 for proposed alternatives. To help the program manager identify and budget for potential manufacturing technology initiatives that could reduce cost.","Additional Considerations: \n• The government program manager, the contractor, and the service-specific ManTech organization need to firm up what investments need to be made to address affordability on the program.\n• Affordability risks include large cost items, items with large variations, and new items with unknown costs.\n• Potential investments might be in material, labor, major NRE (internal and suppliers), tooling, facilities, etc. \n","Lessons Learned: \n• Product affordability has the highest likelihood of being achieved if investments to reduce cost begin during the process of hardware development.\n• Budgeting efforts should be programmed to mature the manufacturing processes and to understand the risk."]
[164,2,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Do the program/projects have reasonable budget estimates for reaching MRL 3 through experiment?","Purpose: \nTo understand the budget available to mitigate the manufacturing risk of proceeding to Milestone A and ensure that a reasonable budget is in place to mature the manufacturing processes to an MRL 6 for proposed alternatives. To help the program manager identify and budget for potential manufacturing technology initiatives that could reduce cost.","Additional Considerations: \n• The government program manager, the contractor, and the service-specific ManTech organization need to firm up what investments need to be made to address affordability on the program.\n• Affordability risks include large cost items, items with large variations, and new items with unknown costs.\n• Potential investments might be in material, labor, major NRE (internal and suppliers), tooling, facilities, etc. \n","Lessons Learned: \n• Product affordability has the highest likelihood of being achieved if investments to reduce cost begin during the process of hardware development.\n• Budgeting efforts should be programmed to mature the manufacturing processes and to understand the risk."]
[165,2,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Have manufacturing investment budget ROM estimates been identified to support industrial base and manufacturing capability gap closure strategies?","Purpose: \nTo understand the budget available to mitigate the manufacturing risk of proceeding to Milestone A and ensure that a reasonable budget is in place to mature the manufacturing processes to an MRL 6 for proposed alternatives. To help the program manager identify and budget for potential manufacturing technology initiatives that could reduce cost.","Additional Considerations: \n• The government program manager, the contractor, and the service-specific ManTech organization need to firm up what investments need to be made to address affordability on the program.\n• Affordability risks include large cost items, items with large variations, and new items with unknown costs.\n• Potential investments might be in material, labor, major NRE (internal and suppliers), tooling, facilities, etc. \n","Lessons Learned: \n• Product affordability has the highest likelihood of being achieved if investments to reduce cost begin during the process of hardware development.\n• Budgeting efforts should be programmed to mature the manufacturing processes and to understand the risk."]
[166,3,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Do the program/projects have reasonable budget estimates for reaching MRL 4 by Milestone (MS) A?","Purpose: \nTo understand the budget available to mitigate the manufacturing risk of proceeding to Milestone A and ensure that a reasonable budget is in place to mature the manufacturing processes to an MRL 6 for proposed alternatives. To help the program manager identify and budget for potential manufacturing technology initiatives that could reduce cost.","Additional Considerations: \n• The government program manager, the contractor, and the service-specific ManTech organization need to firm up what investments need to be made to address affordability on the program.\n• Affordability risks include large cost items, items with large variations, and new items with unknown costs.\n• Potential investments might be in material, labor, major NRE (internal and suppliers), tooling, facilities, etc. \n","Lessons Learned: \n• Product affordability has the highest likelihood of being achieved if investments to reduce cost begin during the process of hardware development.\n• Budgeting efforts should be programmed to mature the manufacturing processes and to understand the risk."]
[167,3,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Have preliminary manufacturing investment budget estimates for manufacturing gap closure recommendations been developed?","Purpose: \nTo ensure the budget is adequate to achieve MRL 6, and to ensure there are mitigation plans in place to address all outstanding issues from initial Manufacturing Readiness Assessments. The program manager needs to assess the funding available to achieve an MRL 6 by the end of TD, and if there is not, to identify the risk of proceeding into EMD. He should build prototypes in a production relevant environment by the end of TD.","Additional Considerations: \n• Does equipment, tooling, or STE at the supplier require investment? If so, who will fund it?\n• Consider leveraging outside investments (e.g., SBIR, IRAD, ManTech, etc.).","Lessons Learned: \n• Product affordability has the highest likelihood of being achieved if investments to reduce cost have begun by this stage."]
[168,4,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Have manufacturing technology initiatives been incorporated to reduce costs?","Purpose: \nTo ensure the budget is adequate to achieve MRL 6, and to ensure there are mitigation plans in place to address all outstanding issues from initial Manufacturing Readiness Assessments. The program manager needs to assess the funding available to achieve an MRL 6 by the end of TD, and if there is not, to identify the risk of proceeding into EMD. He should build prototypes in a production relevant environment by the end of TD.","Additional Considerations: \n• Does equipment, tooling, or STE at the supplier require investment? If so, who will fund it?\n• Consider leveraging outside investments (e.g., SBIR, IRAD, ManTech, etc.).","Lessons Learned: \n• Product affordability has the highest likelihood of being achieved if investments to reduce cost have begun by this stage."]
[169,4,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Does the program have a reasonable manufacturing budget estimate to reach MRL 6 by Milestone (MS) B?","Purpose: \nTo assess the EMD budget to determine if it is reasonable for maturing the manufacturing processes to an MRL 8, and that all manufacturing risk items in TD have appropriate mitigation plans in place. The program manager needs understand what budget is necessary to achieve an MRL 8 at the end of EMD and that there are approved mitigation plans in place to address the risk identified in TD before proceeding into EMD. One of the major requirements for EMD will be to build product in Pilot Line environment, and this requires discussion between the customer and supplier to fully understand funds needed to achieve this. If this is not addressed before entering EMD, it could become a major problem during EMD","\nAdditional Considerations: \n• Does equipment, tooling, or STE at the supplier require investment? If necessary, has this been considered in overall program cost estimates?","\nLessons Learned: \n• Product affordability has the highest likelihood of being achieved if investments to reduce cost have begun before EMD."]
[170,4,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Does the manufacturing budget estimate include capital investment for equipment to be used in the production relevant environment?","Purpose: \nTo assess the EMD budget to determine if it is reasonable for maturing the manufacturing processes to an MRL 8, and that all manufacturing risk items in TD have appropriate mitigation plans in place. The program manager needs understand what budget is necessary to achieve an MRL 8 at the end of EMD and that there are approved mitigation plans in place to address the risk identified in TD before proceeding into EMD. One of the major requirements for EMD will be to build product in Pilot Line environment, and this requires discussion between the customer and supplier to fully understand funds needed to achieve this. If this is not addressed before entering EMD, it could become a major problem during EMD","\nAdditional Considerations: \n• Does equipment, tooling, or STE at the supplier require investment? If necessary, has this been considered in overall program cost estimates?","\nLessons Learned: \n• Product affordability has the highest likelihood of being achieved if investments to reduce cost have begun before EMD."]
[171,4,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Are all outstanding MRL 4 risk areas understood with approved mitigation plans in place?","Purpose: \nTo assess the EMD budget to determine if it is reasonable for maturing the manufacturing processes to an MRL 8, and that all manufacturing risk items in TD have appropriate mitigation plans in place. The program manager needs understand what budget is necessary to achieve an MRL 8 at the end of EMD and that there are approved mitigation plans in place to address the risk identified in TD before proceeding into EMD. One of the major requirements for EMD will be to build product in Pilot Line environment, and this requires discussion between the customer and supplier to fully understand funds needed to achieve this. If this is not addressed before entering EMD, it could become a major problem during EMD","\nAdditional Considerations: \n• Does equipment, tooling, or STE at the supplier require investment? If necessary, has this been considered in overall program cost estimates?","\nLessons Learned: \n• Product affordability has the highest likelihood of being achieved if investments to reduce cost have begun before EMD."]
[172,5,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Does the program have an updated manufacturing budget estimate for reaching MRL 6 by Milestone (MS) B?","Purpose: \nTo use demonstrated results from a production representative environment to ensure production requirements can be met. To use data for updating budgets to achieve an MRL 8 and ensure all manufacturing risks that are identified have approved mitigation plans in place. The program must use actual product build data from a production representative environment to demonstrate confidence in the budget to achieve program maturity requirements. The program needs to use this data to update budgets as required.","Additional Considerations: \n• You need actual build costs compared to original estimates to validate whether return on investment (ROI) is adequate to justify further investment.\n• Costs for subsystems and materials must be applicable to the objective military system design.","Lessons Learned: \n• The need for realism of the cost model requires the use of actual build data. This data will impact the affordability of alternatives being considered.\n• The cost model needs to be populated with actual production representative build data, and it must continually be updated to reflect changes.\n• A program must stay focused on the cost reduction effort at the beginning of EMD, or when you transition into production you will use the inefficiencies in the past to justify the cost for production versus what it should cost."]
[173,5,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Are all outstanding MRL 5 risks and issues understood with approved mitigation plans in place?","Purpose: \nTo use demonstrated results from a production representative environment to ensure production requirements can be met. To use data for updating budgets to achieve an MRL 8 and ensure all manufacturing risks that are identified have approved mitigation plans in place. The program must use actual product build data from a production representative environment to demonstrate confidence in the budget to achieve program maturity requirements. The program needs to use this data to update budgets as required.","Additional Considerations: \n• You need actual build costs compared to original estimates to validate whether return on investment (ROI) is adequate to justify further investment.\n• Costs for subsystems and materials must be applicable to the objective military system design.","Lessons Learned: \n• The need for realism of the cost model requires the use of actual build data. This data will impact the affordability of alternatives being considered.\n• The cost model needs to be populated with actual production representative build data, and it must continually be updated to reflect changes.\n• A program must stay focused on the cost reduction effort at the beginning of EMD, or when you transition into production you will use the inefficiencies in the past to justify the cost for production versus what it should cost."]
[174,6,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Does the program have a reasonable manufacturing budget estimate for reaching MRL 8 by Milestone (MS) C?","Purpose: \nTo use demonstrated data from a pilot line environment to verify production requirements are being met and that all manufacturing risk mitigation plans are working. The program must ensure confidence in the budget by using actual hardware build data from a pilot line environment.","Additional Considerations: \n• Do actual build costs validate whether return on investment (ROI) is adequate to justify further investment?\n• Is the customer manufacturing representative involved and agree to the MRL assessment and the Manufacturing Maturation Plan(s) to mitigate risks?","Lessons Learned: \n• Though the budget may be relying on actual pilot line build data, it must continually be updated to reflect changes. The key is to use the data to make informed program investments."]
[175,6,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Does the manufacturing budget estimate include capital investment for equipment to be used on the pilot line prior to Milestone C? ","Purpose: \nTo use demonstrated data from a pilot line environment to verify production requirements are being met and that all manufacturing risk mitigation plans are working. The program must ensure confidence in the budget by using actual hardware build data from a pilot line environment.","Additional Considerations: \n• Do actual build costs validate whether return on investment (ROI) is adequate to justify further investment?\n• Is the customer manufacturing representative involved and agree to the MRL assessment and the Manufacturing Maturation Plan(s) to mitigate risks?","Lessons Learned: \n• Though the budget may be relying on actual pilot line build data, it must continually be updated to reflect changes. The key is to use the data to make informed program investments."]
[176,6,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Are all outstanding MRL 6 risks and issues understood with approved mitigation plans in place?","Purpose: \nTo use demonstrated data from a pilot line environment to verify production requirements are being met and that all manufacturing risk mitigation plans are working. The program must ensure confidence in the budget by using actual hardware build data from a pilot line environment.","Additional Considerations: \n• Do actual build costs validate whether return on investment (ROI) is adequate to justify further investment?\n• Is the customer manufacturing representative involved and agree to the MRL assessment and the Manufacturing Maturation Plan(s) to mitigate risks?","Lessons Learned: \n• Though the budget may be relying on actual pilot line build data, it must continually be updated to reflect changes. The key is to use the data to make informed program investments."]
[177,7,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Does the program have an updated manufacturing budget estimate for reaching MRL 8 by Milestone (MS) C?","Purpose: \nTo use demonstrated results from a LRIP line to verify an adequate budget exists to achieve production rate requirements and that all manufacturing risks areas have approved mitigation plans. The program manager must verify the realism of the budget by using actual build data from a LRIP environment to ensure confidence in meeting production requirements. This early production data will set the foundation for future production efforts and management needs to understand where emphasis must be placed to reduce cost and ensure production objectives can be achieved.","Additional Considerations: \n• Is actual cost data from LRIP builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of FRP manufacturing.","Lessons Learned: \n• You must continually update budgets as more data becomes available and use this data to ensure program objectives can be met."]
[178,7,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Are all outstanding MRL 7 risks and issues understood with approved mitigation plans in place?","Purpose: \nTo use demonstrated results from a LRIP line to verify an adequate budget exists to achieve production rate requirements and that all manufacturing risks areas have approved mitigation plans. The program manager must verify the realism of the budget by using actual build data from a LRIP environment to ensure confidence in meeting production requirements. This early production data will set the foundation for future production efforts and management needs to understand where emphasis must be placed to reduce cost and ensure production objectives can be achieved.","Additional Considerations: \n• Is actual cost data from LRIP builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of FRP manufacturing.","Lessons Learned: \n• You must continually update budgets as more data becomes available and use this data to ensure program objectives can be met."]
[179,8,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Does the program have a reasonable manufacturing budget estimate for reaching MRL 9 by the Full Rate Production (FRP) decision point?","Purpose: \nTo verify you have an adequate budget to achieve you production rate requirements. The program manager must continually verify the realism of the budget by using actual build data from a production environment to ensure confidence in meeting production requirements.","Additional Considerations: \n• Is actual cost data from FRP builds being used?\n• Include updated costs from suppliers including yield/rework which might affect cost of FRP manufacturing.","Lessons Learned: \n• You must continually update budgets as more data becomes available and use this data to ensure program objectives can be met."]
[180,8,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Does the manufacturing budget estimate include investments for Low Rate Initial Production (LRIP) and Full Rate Production (FRP)?","Purpose: \nTo begin to identify material properties identified for research, and use this information to allow decision makers insight into potential investments to address those properties. Researchers must begin understanding and developing material properties needed to provide future capability to the war fighter. \n","Additional Considerations: \n• Consider historically known material limitations and compatibility issues.","Lessons Learned: \n• Historically materials manufacturability and producibility issues are ignored early in R&D so identifying potential materials manufacturability issues at this point will reduce effort in later stages of the product life cycle."]
[181,8,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Are all outstanding MRL 8 risks and issues understood with approved mitigation plans in place?","Purpose: \nTo begin predicting material properties and characteristics from research being performed. S&T organizations such as service laboratories and research organizations in academia and industry must begin working with the acquisition community at this time. If it is anticipated that there will be hardware migrating to the next phase there should be some effort to predict the material properties and characteristics anticipated before completing the R&D effort.\n","Additional Considerations: \n• Consider historically known material limitations and compatibility issues. \n• Will new materials or product development concepts most likely be required?\n• Is a new technology or process available to process the material?\n","Lessons Learned: \n• Historically materials manufacturability and producibility issues are ignored early in R&D. \n• This is a good time to identify materials producibility and manufacturability risks to the R&D community before transitioning so they can understand potential risk of making the product."]
[182,9,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Does the program have a reasonable manufacturing budget estimate for Full Rate Production (FRP)?","Purpose: \nTo assess the properties of the materials being proposed to transition into acquisition. To effectively transition R&D projects one must address the manufacturing maturity of the materials being developed.","Additional Considerations: \n• Consider historically known material limitations and compatibility issues. \n• Consider manufacturing issues in life-cycle cost, quality, and cycle time from literature, simulation or technology demonstration.","Lessons Learned: \n• Materials producibility and manufacturability risks must be presented to the to the R&D community before transitioning so they can understand potential risk of making the product."]
[183,9,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Are all outstanding MRL 9 risks and issues understood with approved mitigation plans in place?","Purpose: \nTo assess the maturity of proposed materials to understand manufacturing risk to program. The materials produced in a laboratory environment should be assessed to determine additional steps needed to achieve program requirements.","Additional Considerations: \n• Consider materials use in similar products or technology demonstrators and lessons learned from each.\n• New materials maybe new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n","Lessons Learned: \n• Materials producibility and manufacturability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives."]
[184,10,"C - Cost & Funding","C.3 - Manufacturing Investment Budget","Is the production budget sufficient for production at the required rates and schedule to support the funded program?","Purpose: \nTo understand how the materials are being produced for prototypes and compare those results to what is required to achieve program objectives. To do initial assessment of materials in a production relevant environment. Use this information to put into place the appropriate mitigation plans to address all outstanding issues/risks for maturity your materials. The program manager must assess material maturity in the earliest acquisition phases.\n","Additional Considerations: \n• Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.","\n\nLessons Learned: \n• Materials producibility and manufacturability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives.\n• Develop a plan to address material maturity risk as the program goes through TD and transitions into EMD."]
[185,1,"D - Materials","D.1 - Maturity","Have new material properties and characteristics been surveyed and identified for research (e.g., manufacturability, quality)?","Purpose: \nTo understand how the materials are being produced for prototypes and compare those results to what is required to achieve program objectives. To do initial assessment of materials in a production relevant environment. Use this information to put into place the appropriate mitigation plans to address all outstanding issues/risks for maturity your materials. The program manager must assess material maturity in the earliest acquisition phases.\n","Additional Considerations: \n• Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.","\n\nLessons Learned: \n• Materials producibility and manufacturability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives.\n• Develop a plan to address material maturity risk as the program goes through TD and transitions into EMD."]
[186,2,"D - Materials","D.1 - Maturity","Have the potential effects of new material properties on design application manufacturability and quality been predicted based on research?"]
[187,3,"D - Materials","D.1 - Maturity","Have the effects of new material properties on design concept manufacturability and quality been validated using experiments and models?","Purpose: \nTo assess the material maturity demonstrated in the TMRR phase versus what is required to achieve program objectives. To have adequate material specifications in place to transition to EMD and ensure that the material properties have been adequately assessed with appropriate mitigation plans in place to proceed to EMD. The program manager must understand the material maturity achieved from producing the prototype units in TMRR and the risk of proceeding to EMD.","\nAdditional Considerations: \n• Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.","\n\nLessons Learned: \n• Execute the plan to address material maturity risk as the program goes through TD and transitions into EMD."]
[188,4,"D - Materials","D.1 - Maturity","Have new materials and components for the preferred materiel solution been demonstrated in a laboratory environment?","Purpose: \nTo assess the material maturity demonstrated in the TMRR phase versus what is required to achieve program objectives. To have adequate material specifications in place to transition to EMD and ensure that the material properties have been adequately assessed with appropriate mitigation plans in place to proceed to EMD. The program manager must understand the material maturity achieved from producing the prototype units in TMRR and the risk of proceeding to EMD.","\nAdditional Considerations: \n• Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.","\n\nLessons Learned: \n• Execute the plan to address material maturity risk as the program goes through TD and transitions into EMD."]
[189,5,"D - Materials","D.1 - Maturity","Have all materials been manufactured or produced in a prototype environment (e.g., production relevant)?","Purpose: \nTo assess the material maturity demonstrated in the TMRR phase versus what is required to achieve program objectives. To have adequate material specifications in place to transition to EMD and ensure that the material properties have been adequately assessed with appropriate mitigation plans in place to proceed to EMD. The program manager must understand the material maturity achieved from producing the prototype units in TMRR and the risk of proceeding to EMD.","\nAdditional Considerations: \n• Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.","\n\nLessons Learned: \n• Execute the plan to address material maturity risk as the program goes through TD and transitions into EMD."]
[190,5,"D - Materials","D.1 - Maturity","Were any materials manufactured or produced a similar application and/or program?","Purpose: \nTo use demonstrated results from a production representative environment to verify material specifications are sufficiently mature to achieve production requirements.","Additional Considerations: \n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.","Lessons Learned: \n• Material maturity must be continually assessed in this program phase, and the focus will be on transition to production."]
[191,5,"D - Materials","D.1 - Maturity","Are material maturation efforts in place to address risks and issues from producing new materials for the technology demonstration?","Purpose: \nTo use demonstrated results from a production representative environment to verify material specifications are sufficiently mature to achieve production requirements.","Additional Considerations: \n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.","Lessons Learned: \n• Material maturity must be continually assessed in this program phase, and the focus will be on transition to production."]
[192,6,"D - Materials","D.1 - Maturity","Have technology demonstration articles (e.g., prototypes) verified maturity of materials?","Purpose: \nTo use demonstrated data from a pilot line environment to verify material specifications are mature and sufficient to achieve production requirements for LRIP.","Additional Considerations: \n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.","Lessons Learned: \n• Material maturity must be continually assessed in this program phase, and the focus will be on transition to production (i.e. LRIP)."]
[193,6,"D - Materials","D.1 - Maturity","Are preliminary material specifications in place?","Purpose: \nTo use demonstrated data from a pilot line environment to verify material specifications are mature and sufficient to achieve production requirements for LRIP.","Additional Considerations: \n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.","Lessons Learned: \n• Material maturity must be continually assessed in this program phase, and the focus will be on transition to production (i.e. LRIP)."]
[194,6,"D - Materials","D.1 - Maturity","Have material properties been adequately characterized to enable producing prototypes in a production relevant environment?","Purpose: \nTo assure all material is being controlled by a specification in LRIP. To use the demonstrated results from a LRIP line to verify adequate maturity to achieve production rate requirements and that all manufacturing risk areas have approved mitigation plans. Materials should have achieved maturity by this phase.","Additional Considerations: \n• Are new materials being introduced at this stage of the program? If so, you need to carefully assess whether potential impacts to production might outweigh the apparent benefit of any new materials.","Lessons Learned: \n• Maturity of material specifications must be continually evaluated. This area historically represents a significant amount of risk and must be closely monitored."]
[195,7,"D - Materials","D.1 - Maturity","Is the material maturity sufficient for a pilot line build?","Purpose: \nTo assure all material is being controlled by a specification in LRIP. To use the demonstrated results from a LRIP line to verify adequate maturity to achieve production rate requirements and that all manufacturing risk areas have approved mitigation plans. Materials should have achieved maturity by this phase.","Additional Considerations: \n• Are new materials being introduced at this stage of the program? If so, you need to carefully assess whether potential impacts to production might outweigh the apparent benefit of any new materials.","Lessons Learned: \n• Maturity of material specifications must be continually evaluated. This area historically represents a significant amount of risk and must be closely monitored."]
[196,7,"D - Materials","D.1 - Maturity","Have the necessary material specifications been approved?","Purpose: \nTo verify the material specifications are under control as the program achieves production rate requirements. The program must constantly assess material maturity in achieving production requirements.","Additional Considerations: \n• None","Lessons Learned: \n• It is crucial to continually assess material maturity data to ensure program objectives will be met."]
[197,8,"D - Materials","D.1 - Maturity","Have the materials been proven and validated during Engineering and Manufacturing Development (EMD) as adequate to support Low Rate Initial Production (LRIP)?","Purpose: \nTo begin assessing the availability of the materials required. The S&T community needs to begin assessing the availability of the material for their programs and to start addressing any issues identified. If it is anticipated that there will be hardware migrating to the next phase there should be some effort to predict the material availability issues before completing the R&D effort.","Additional Considerations: \n• Consider historically known material limitations and compatibility issues. \n• Will new materials or product development concepts most likely be required?\n• Is a new technology or process available to process the material?","Lessons Learned: \n• Historically materials manufacturability and producibility issues are ignored early in R&D. \n• This is a good time to identify materials producibility and manufacturability risks to the R&D community before transitioning so they can understand potential risk of making the product."]
[198,8,"D - Materials","D.1 - Maturity","Are material specifications stable (i.e., no changes to specifications)?","Purpose: \nTo assess the material availability of projects being proposed to transition into acquisition. To effectively transition S&T projects one must address the availability issues of the materials being developed for R&D projects being considered for acquisition.","Additional Considerations: \n• Consider historically known material limitations and compatibility issues. \n• Will new materials or product development concepts most likely be required?\n• Is a new technology or process available to process the material?\n","Lessons Learned: \n• Historically materials manufacturability and producibility issues are ignored early in R&D where we start transitioning hardware from the labs to acquisition. \n• This is a good time to identify materials producibility and manufacturability risks to the R&D community before transitioning so they can understand potential risk of making the product and assessing the material availability."]
[199,9,"D - Materials","D.1 - Maturity","Are all materials meeting specifications during Low Rate Initial Production (LRIP)? ","Purpose: \nTo assess the availability of proposed materials to understand the manufacturing risk to the program. The lead times for proposed materials for the program should be identified at this stage. Those with long lead times need to be closely monitored as the program transitions into acquisition.\n","Additional Considerations: \n• Consider historically known material limitations and compatibility issues. \n• Will new materials or product development concepts most likely be required?\n• Is a new technology or process available to process the material?\n","Lessons Learned: \n• Hardware availability must be closely monitored as the program transitions from the R&D phase to Acquisition, and it is important to understand and assess lead times of critical materials needed to support the TD phase."]
[200,9,"D - Materials","D.1 - Maturity","Have the materials been proven and validated as adequate to support FRP?","Purpose: \nTo assess the availability of proposed materials to understand the manufacturing risk to the program. The lead times for proposed materials for the program should be identified at this stage. Those with long lead times need to be closely monitored as the program transitions into acquisition.\n","Additional Considerations: \n• Consider historically known material limitations and compatibility issues. \n• Will new materials or product development concepts most likely be required?\n• Is a new technology or process available to process the material?\n","Lessons Learned: \n• Hardware availability must be closely monitored as the program transitions from the R&D phase to Acquisition, and it is important to understand and assess lead times of critical materials needed to support the TD phase."]
[201,10,"D - Materials","D.1 - Maturity","Are all materials meeting specifications during Full Rate Production (FRP)?","Purpose: \nTo determine if material will be available to produce the prototype units. To use this information to put into place appropriate mitigation plans to address all outstanding issues/risks for materials availability.","\nAdditional Considerations: \n• Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).","\n\nLessons Learned: \n• Materials availability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives.\n• Develop a plan to address material availability risk as the program goes through TD and transitions into EMD."]
[202,1,"D - Materials","D.2 - Availability","Have global trends for material availability, obsolescence, and DMSMS been surveyed and identified for research?","Purpose: \nTo determine if material will be available to produce the prototype units. To use this information to put into place appropriate mitigation plans to address all outstanding issues/risks for materials availability.","\nAdditional Considerations: \n• Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).","\n\nLessons Learned: \n• Materials availability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives.\n• Develop a plan to address material availability risk as the program goes through TD and transitions into EMD."]
[203,2,"D - Materials","D.2 - Availability","Has material availability, obsolescence, and DMSMS gaps been identified?","Purpose: \nTo determine if material will be available to produce the prototype units. To use this information to put into place appropriate mitigation plans to address all outstanding issues/risks for materials availability.","\nAdditional Considerations: \n• Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).","\n\nLessons Learned: \n• Materials availability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives.\n• Develop a plan to address material availability risk as the program goes through TD and transitions into EMD."]
[204,3,"D - Materials","D.2 - Availability","Has material availability, obsolescence, and DMSMS gap closure strategy been defined?"]
[205,4,"D - Materials","D.2 - Availability","Have projected lead times been identified for all difficult-to-obtain, difficult-to-process, or hazardous materials?","Purpose: \nTo assess the material availability for the EMD phase and understand the risk. To begin addressing potential DMSMS (Diminishing Manufacturing Sources and Materials Shortages) and obsolescence risk of the materials being proposed. The program manager must understand the availability of the material required to produce the EMD units and the associated risk. Addressing DMSMS issues should begin as early as possible in the design process where there are options to address these issues","Additional Considerations: \n• Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n• Has the program office planned and budgeted for DMSMS?\n","\nLessons Learned: \n• Materials availability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives.\n• Develop a plan to address material availability risk as the program goes through TD and transitions into EMD. \n• Addressing DMSMS issues must begin as early as possible. Often a program has not budgeted to address DMSMS.\n• DMSMS is difficult to address, and to do it adequately, it must be done in the early stages of the process before the design matures."]
[206,4,"D - Materials","D.2 - Availability","Have material quantities and lead times been estimated?","Purpose: \nTo assess the material availability for the EMD phase and understand the risk. To begin addressing potential DMSMS (Diminishing Manufacturing Sources and Materials Shortages) and obsolescence risk of the materials being proposed. The program manager must understand the availability of the material required to produce the EMD units and the associated risk. Addressing DMSMS issues should begin as early as possible in the design process where there are options to address these issues","Additional Considerations: \n• Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n• Has the program office planned and budgeted for DMSMS?\n","\nLessons Learned: \n• Materials availability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives.\n• Develop a plan to address material availability risk as the program goes through TD and transitions into EMD. \n• Addressing DMSMS issues must begin as early as possible. Often a program has not budgeted to address DMSMS.\n• DMSMS is difficult to address, and to do it adequately, it must be done in the early stages of the process before the design matures."]
[207,4,"D - Materials","D.2 - Availability","Have material availability risks and issues for the preferred materiel solution been considered in AoA?","Purpose: \nTo assess the material availability for the EMD phase and understand the risk. To begin addressing potential DMSMS (Diminishing Manufacturing Sources and Materials Shortages) and obsolescence risk of the materials being proposed. The program manager must understand the availability of the material required to produce the EMD units and the associated risk. Addressing DMSMS issues should begin as early as possible in the design process where there are options to address these issues","Additional Considerations: \n• Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n• Has the program office planned and budgeted for DMSMS?\n","\nLessons Learned: \n• Materials availability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives.\n• Develop a plan to address material availability risk as the program goes through TD and transitions into EMD. \n• Addressing DMSMS issues must begin as early as possible. Often a program has not budgeted to address DMSMS.\n• DMSMS is difficult to address, and to do it adequately, it must be done in the early stages of the process before the design matures."]
[208,4,"D - Materials","D.2 - Availability","Have mitigation plans been incorporated in SEP for the preferred system concept?","Purpose: \nTo assess the material availability for the EMD phase and understand the risk. To begin addressing potential DMSMS (Diminishing Manufacturing Sources and Materials Shortages) and obsolescence risk of the materials being proposed. The program manager must understand the availability of the material required to produce the EMD units and the associated risk. Addressing DMSMS issues should begin as early as possible in the design process where there are options to address these issues","Additional Considerations: \n• Materials may be new to the industry, new to the product/program, new to the supplier, critical to the program, or present risk due to availability/compatibility/criticality.\n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n• Has the program office planned and budgeted for DMSMS?\n","\nLessons Learned: \n• Materials availability risks must be presented to the acquisition community before transitioning to understand the overall risk of achieving program objectives.\n• Develop a plan to address material availability risk as the program goes through TD and transitions into EMD. \n• Addressing DMSMS issues must begin as early as possible. Often a program has not budgeted to address DMSMS.\n• DMSMS is difficult to address, and to do it adequately, it must be done in the early stages of the process before the design matures."]
[209,5,"D - Materials","D.2 - Availability","Have material availability risks and issues been addressed for the prototype build? "]
[210,5,"D - Materials","D.2 - Availability","Have all materials been analyzed for availability risks and issues?","Purpose: \nTo look forward to see if there are any issues for material availability for LRIP. To start putting into place strategies to minimize DMSMS issues. The program needs to consistently monitor material availability and work DMSMS issues.","Additional Considerations: \n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n• Has the program office planned and budgeted for DMSMS?","Lessons Learned: \n• The program manager must continually assess material availability and manage DMSMS issues as the program moves into production."]
[211,5,"D - Materials","D.2 - Availability","Are there any significant material availability risks?","Purpose: \nTo look forward to see if there are any issues for material availability for LRIP. To start putting into place strategies to minimize DMSMS issues. The program needs to consistently monitor material availability and work DMSMS issues.","Additional Considerations: \n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n• Has the program office planned and budgeted for DMSMS?","Lessons Learned: \n• The program manager must continually assess material availability and manage DMSMS issues as the program moves into production."]
[212,5,"D - Materials","D.2 - Availability","Has planning been initiated to address availability for scale-up?","Purpose: \nTo look forward to see if there are any issues for material availability for LRIP. To start putting into place strategies to minimize DMSMS issues. The program needs to consistently monitor material availability and work DMSMS issues.","Additional Considerations: \n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n• Has the program office planned and budgeted for DMSMS?","Lessons Learned: \n• The program manager must continually assess material availability and manage DMSMS issues as the program moves into production."]
[213,6,"D - Materials","D.2 - Availability","Have material availability risks and issues been addressed to meet Engineering and Manufacturing Development (EMD) build?","Purpose: \nTo assess and address the material availability for LRIP and begin reviewing material issues for FRP.","Additional Considerations: \n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n• Has the program office planned and budgeted for DMSMS?","Lessons Learned: \n• The program manager must continually assess material availability and manage DMSMS issues as the program moves into production (LRIP and FRP)."]
[214,6,"D - Materials","D.2 - Availability","Have materials, items, components, etc. been identified which require a long lead time to obtain?","Purpose: \nTo assess and address the material availability for LRIP and begin reviewing material issues for FRP.","Additional Considerations: \n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n• Has the program office planned and budgeted for DMSMS?","Lessons Learned: \n• The program manager must continually assess material availability and manage DMSMS issues as the program moves into production (LRIP and FRP)."]
[215,6,"D - Materials","D.2 - Availability","Have materials, items, components, etc. been identified which are at risk of obsolescence during the product life cycle?","Purpose: \nTo assess and address the material availability for LRIP and begin reviewing material issues for FRP.","Additional Considerations: \n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).\n• Has the program office planned and budgeted for DMSMS?","Lessons Learned: \n• The program manager must continually assess material availability and manage DMSMS issues as the program moves into production (LRIP and FRP)."]
[216,6,"D - Materials","D.2 - Availability","Have materials, items, components, etc. been identified which are at risk from Diminishing Manufacturing Sources and Material Shortages during the product life cycle? ","Purpose: \nTo assess the material availability for FRP and ensure there are no significant risks.","Additional Considerations: \n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).","Lessons Learned: \n• The program manager must continually assess material availability and manage DMSMS issues as the program moves into FRP."]
[217,7,"D - Materials","D.2 - Availability","Have material availability risks and issues been addressed to meet Low Rate Initial Production (LRIP) builds?","Purpose: \nTo assess the material availability for FRP and ensure there are no significant risks.","Additional Considerations: \n• Consider lead times, obsolescence, new suppliers where availability is not proven, sole/ foreign sources, volume rates that are higher than typical, ITAR, REACH concerns, etc.\n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).","Lessons Learned: \n• The program manager must continually assess material availability and manage DMSMS issues as the program moves into FRP."]
[218,7,"D - Materials","D.2 - Availability","Have materials, items, components, etc. which require a long lead time been identified and mitigated for Low Rate Initial Production (LRIP)?","Purpose: \nTo assess the material availability for FRP and make sure there are no significant risks.","Additional Considerations: \n• Consider issues associated with shipment and control of COMSEC units (affects shipping times).","Lessons Learned: \n• The program manager must continually assess material availability and manage DMSMS issues as the program is in FRP."]
[219,7,"D - Materials","D.2 - Availability","Is a plan in place for materials, items, components, etc. that are identified as at risk of obsolescence during the product life cycle?","Purpose: \nTo evaluate whether or not there are potential suppliers that are capable of addressing the required technology as well as material solutions of the customer requirements. Since 70 to 80 percent of the cost and risk of any weapon system is in the supply chain, an assessment of the potential supply chain capability must be conducted during this phase. An assessment of supply chain capability identifies the needs of the program versus the established potential suppliers. Contractors need to establish very early if they need to build time into the schedule to develop a particular skill set, processes, or material in-house or within the supply base to meet program requirements.\n","Additional Considerations: \n• Are there new manufacturing processes needed to produce the design?\n• Have lists been developed to identify new equipment, tooling or test equipment that may be needed to produce the design?\n• Have the make/buy decisions been initiated?\n• Can the industrial base support Technology Development (TD) concepts?\n• Have potential supplier survey topics been identified? \n","Lessons Learned: \n• Gain a macro level view of the financial viability of the supply base. Just because a supplier is technically able to perform the work does not mean that they will be around long enough to provide the product if their financial position is not sound. \n• Attend as many design reviews as possible to gain an understanding of the product and its unique features. Make a list of these features/design concerns, and have the prime contractor address each concern to understand if the design is producible."]
[220,7,"D - Materials","D.2 - Availability","Are mitigation strategies in place for materials, items, components, etc. that are identified as at risk from Diminishing Manufacturing Sources and Material Shortages during the product life cycle?","Purpose: \nTo do an assessment for potential supply chain sources is in effect a manufacturing feasibility evaluation to determine: (1) Identification of manufacturing technologies and processes needed; (2) Materials/processes needed for producibility; (3) Potential tradeoff in design options and supply chain ability to support these options; and (4) Supply chain industrial base capabilities as well as potential gaps/risks. During this phase of the acquisition cycle it is important to assess the supply chain industrial base capabilities by completing a supply chain survey to identify gaps/ risks for the preferred concepts and key technology components as well as potential key processes. A survey of supply chain capability indicates that the prime contractor has done the necessary research to understand the potential suppliers for this particular product/program and and/or identified the gaps that exist and mitigation plans to close those gaps.","Additional Considerations: \n• Is the prime satisfied that the proposed design can be manufactured, assembled, tested, packaged, and delivered in sufficient quantity, at an acceptable cost to the customer on schedule with the identified potential supply base?\n• Has the prime established criteria for selection and evaluation of its suppliers?\n• Has the prime conducted an on-site assessment of potential suppliers? When?\n• Critical/new suppliers may need to be tracked more carefully due to potential program risk.\n","Lessons Learned: \n• Open communication between the government, prime contractor and the supply base throughout the program execution is critical to success.\n• Review and consider past performance data on supplier capabilities and consider “similar to” supply chains\n• One needs to discuss and understand where the engineering and technical support is located for the manufacturing facility and how that support communicates to the manufacturing location."]
[221,8,"D - Materials","D.2 - Availability","Are all materials availability risks and issues being managed (i.e., mitigations plans being executed) for Low Rate Initial Production (LRIP)?","Purpose: \nTo ensure suppliers can produce prototype components in a production relevant environment. To identify potential supply chain sources and evaluate their capability to support prototype build. To initiate an industrial base capability assessment to identify potential supply chain manufacturing sources (i.e. sole/single/foreign source suppliers and suppliers of technologies with potential obsolescence issues). Although it is not expected that suppliers would have a complete factory and supply chain established this early in a program, it is necessary to obtain some baseline knowledge on critical manufacturing processes, potential production scale up efforts, and potential supply chain issues. The bottom line is potential supply chain sources must be identified and capable to support prototype build by either having or planning to obtain the ability, technical expertise, capacity, willingness, manpower and equipment to fully support build of the designed product.","\nAdditional Considerations:\n• Does a formal and effective sub-tier supplier management process exist?\n• Are there preliminary assessments within the supply chain highlighting any concerns relative to tooling and/or testing that may impact prototype build?","\nLessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier on what represents a prototype production for your programs. \n• Understanding the long lead material, equipment, and resource situation are key to understanding where risk potentially lies in the prototype plan. An understanding of these elements will help direct focus in the appropriate direction to provide for a successful prototype program. \n• Some type of formal communication of expectations and requirements that must be flowed down to the supply chain. There must also be a channel in which the supply chain can flow up questions and concerns."]
[222,8,"D - Materials","D.2 - Availability","Have procurements for all long lead materials been initiated for initial production (LRIP)?","Purpose : \nTo develop: (1) An updated critical supplier list; (2) Life cycle supply chain requirement (i.e. single source/ sole source/ foreign sources, suppliers with long lead items, suppliers with potential DMS issues) and (3) Updated supply chain plans (e.g. teaming agreements, Corporate Leverage Contracting (CLC), etc.) in place leading to an EMD contract award. During the TD phase an examination of the producibility of a proposed design is needed to allow for trade analysis decisions to be made on critical suppliers’ cost, performance and schedule. It is significantly easier to make changes during this phase where changes potentially have greater impact on key performance metrics and life cycle cost. Given that 70 to 80 percent of the BOM is in the supply chain, suppliers contribute immensely in this area. Lifecycle supply chain requirements refer to all phases of the intended acquisition category being understood and relayed to the supply chain.","Additional Considerations:\n• Is there capacity at each of the identified suppliers to take on this workload?\n• Is there an understanding and ranking of the most “critical” suppliers, for quality and delivery?\n• Is there a list of supplier capability/capacity requirements and goals?\n• Have Critical/new supplier been identified for risk documentation?","\n\n\n\nLessons Learned: \n• Understand why a certain supplier is chosen for a particular part. Ask if there has been similar product produced at that supplier and received by the consuming plant. Ask about the working relationships between the parties.\n• Critical suppliers should understand the program schedule and be able to talk to any areas that may be a problem. \n• Understand the supplier selection criteria as well as their past performance if available."]
[223,8,"D - Materials","D.2 - Availability","Are materials availability risks and issues being addressed (i.e., mitigation plans) to meet FRP builds?","Purpose : \nTo develop: (1) An updated critical supplier list; (2) Life cycle supply chain requirement (i.e. single source/ sole source/ foreign sources, suppliers with long lead items, suppliers with potential DMS issues) and (3) Updated supply chain plans (e.g. teaming agreements, Corporate Leverage Contracting (CLC), etc.) in place leading to an EMD contract award. During the TD phase an examination of the producibility of a proposed design is needed to allow for trade analysis decisions to be made on critical suppliers’ cost, performance and schedule. It is significantly easier to make changes during this phase where changes potentially have greater impact on key performance metrics and life cycle cost. Given that 70 to 80 percent of the BOM is in the supply chain, suppliers contribute immensely in this area. Lifecycle supply chain requirements refer to all phases of the intended acquisition category being understood and relayed to the supply chain.","Additional Considerations:\n• Is there capacity at each of the identified suppliers to take on this workload?\n• Is there an understanding and ranking of the most “critical” suppliers, for quality and delivery?\n• Is there a list of supplier capability/capacity requirements and goals?\n• Have Critical/new supplier been identified for risk documentation?","\n\n\n\nLessons Learned: \n• Understand why a certain supplier is chosen for a particular part. Ask if there has been similar product produced at that supplier and received by the consuming plant. Ask about the working relationships between the parties.\n• Critical suppliers should understand the program schedule and be able to talk to any areas that may be a problem. \n• Understand the supplier selection criteria as well as their past performance if available."]
[224,9,"D - Materials","D.2 - Availability","Have long lead procurements been initiated for Full Rate Production (FRP)?"]
[225,9,"D - Materials","D.2 - Availability","Are all materials availability risks and issues being managed (i.e., mitigations plans being executed) for Low Rate Initial Production (LRIP)? ","Purpose: \nFor the prime to maintain: (1) Well defined and documented supply chain management processes and procedures; (2) An initial plan to develop a set of supplier predictive indicators; and (3) Assessment results from first tier critical suppliers (e.g. capability, capacity, etc.). Management of the supply chain has become critical to achieving program cost, schedule, and performance objectives. The percentage of work performed at the suppliers and the risk they present to the program is significant, and to adequately manage a program requires insight into how suppliers are performing and ensuring they are adequately managed.","Additional Considerations:\n• Has an adequate assessment of the first tier supply chain been completed?\n• Is management aware of supplier performance metrics and ready to take action if needed?\n• How are “high risk” suppliers managed? Is it different than less risky suppliers?\n• Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n• Does the contractor have an adequate process in place to identify and ensure program requirements are properly flowed down to supplier/subcontractors?\n• Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n• Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n• What criteria are used to determine the capability/capacity of subcontractors and critical vendors/suppliers?\n• Identify the major/critical subcontractors/vendors for this program.\n• Are subcontractor/vendor delivery schedules compatible with production schedules?\n• How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?","Lessons Learned: \n• The problems in supplier management are the same in dealing with the prime: Not identifying the flowing down the right requirements, not selecting the best supplier to effectively achieve your requirements, supplier performance not adequately monitored to spot problems early, supplier requirements are not adequately integrated into an overall integrated program schedule.\n• Review tracking of supplier performance metrics by senior management. Is the tracking reviewed regularly at the appropriate management level? Have any non-conformances or late deliveries been followed up on at the root cause/ corrective action level?\n• The key to proper assessment of suppliers is not the set of audit questions; it is the conduct of the audit itself and the follow up actions.\n• Review risk mitigation plans. Are the mitigation actions sufficient to mitigate documented risks? Is there a feedback loop to ensure mitigation actions are completed?\n• High-risk suppliers need to be managed differently than less risky suppliers."]
[226,10,"D - Materials","D.2 - Availability","Are all materials availability risks and issues being managed for FRP?"]
[227,1,"D - Materials","D.3 - Supply Chain Management","Have global trends for supply chain capability and capacity been surveyed?","Purpose: \nFor the prime to maintain: (1) Well defined and documented supply chain management processes and procedures; (2) An initial plan to develop a set of supplier predictive indicators; and (3) Assessment results from first tier critical suppliers (e.g. capability, capacity, etc.). Management of the supply chain has become critical to achieving program cost, schedule, and performance objectives. The percentage of work performed at the suppliers and the risk they present to the program is significant, and to adequately manage a program requires insight into how suppliers are performing and ensuring they are adequately managed.","Additional Considerations:\n• Has an adequate assessment of the first tier supply chain been completed?\n• Is management aware of supplier performance metrics and ready to take action if needed?\n• How are “high risk” suppliers managed? Is it different than less risky suppliers?\n• Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n• Does the contractor have an adequate process in place to identify and ensure program requirements are properly flowed down to supplier/subcontractors?\n• Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n• Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n• What criteria are used to determine the capability/capacity of subcontractors and critical vendors/suppliers?\n• Identify the major/critical subcontractors/vendors for this program.\n• Are subcontractor/vendor delivery schedules compatible with production schedules?\n• How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?","Lessons Learned: \n• The problems in supplier management are the same in dealing with the prime: Not identifying the flowing down the right requirements, not selecting the best supplier to effectively achieve your requirements, supplier performance not adequately monitored to spot problems early, supplier requirements are not adequately integrated into an overall integrated program schedule.\n• Review tracking of supplier performance metrics by senior management. Is the tracking reviewed regularly at the appropriate management level? Have any non-conformances or late deliveries been followed up on at the root cause/ corrective action level?\n• The key to proper assessment of suppliers is not the set of audit questions; it is the conduct of the audit itself and the follow up actions.\n• Review risk mitigation plans. Are the mitigation actions sufficient to mitigate documented risks? Is there a feedback loop to ensure mitigation actions are completed?\n• High-risk suppliers need to be managed differently than less risky suppliers."]
[228,2,"D - Materials","D.3 - Supply Chain Management","Have potential supply chain capability and capacity gaps been identified?","Purpose: \nTo ensure: (1) Supply chain adequacy to support LRIP; (2) Adequate assessment of critical second and lower tier supply chain capability; (3) That proper processes and discipline are in place to ensure effective supplier requirements are being flowed down. Most prime contractors rely on their first tier suppliers to manage the lower tier. This methodology would be adequate if the prime could be certain that the first tier and second tier critical suppliers have the proper discipline to manage lower tier sub-contractors. Some primes perform Process Validation Audits (PVAs) on their first tier critical suppliers to ensure proper sub-tier management. Assessment of critical second and lower tier supply chain evidence assures the customer that the contractor has performed a thorough review of the critical manufacturing processes needed to produce the intended design at the supplier.","Additional Considerations:\n• Has an adequate assessment of the first tier supply chain been completed?\n• Is management aware of supplier performance metrics and ready to take action if needed?\n• How are “high risk” suppliers managed? Is it different than less risky suppliers?\n• Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n• Does the contractor have an adequate process in place to identify and ensure program requirements are properly flowed down to supplier/subcontractors?\n• Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n• Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n• What criteria are used to determine the capability/capacity of subcontractors and critical vendors/suppliers?\n• Identify the major/critical subcontractors/vendors for this program.\n• Are subcontractor/vendor delivery schedules compatible with production schedules?\n• How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?","Lessons Learned: \n• Look for a feedback loop to indicate that flow-down requirements are understood and verified. It is NOT sufficient for prime to communicate requirements without confirming understanding and acceptance.\n• Implementing a proactive and robust requirements flow-down process is a challenge. An undefined change control process is another area that can be a source of risk for the program. \n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area. \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality. Suppliers may not understand the application of their products into the overall system. The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
[229,3,"D - Materials","D.3 - Supply Chain Management","Have supply chain capability and capacity gap closure strategies been defined?","Purpose: \nTo ensure: (1) Supply chain adequacy to support LRIP; (2) Adequate assessment of critical second and lower tier supply chain capability; (3) That proper processes and discipline are in place to ensure effective supplier requirements are being flowed down. Most prime contractors rely on their first tier suppliers to manage the lower tier. This methodology would be adequate if the prime could be certain that the first tier and second tier critical suppliers have the proper discipline to manage lower tier sub-contractors. Some primes perform Process Validation Audits (PVAs) on their first tier critical suppliers to ensure proper sub-tier management. Assessment of critical second and lower tier supply chain evidence assures the customer that the contractor has performed a thorough review of the critical manufacturing processes needed to produce the intended design at the supplier.","Additional Considerations:\n• Has an adequate assessment of the first tier supply chain been completed?\n• Is management aware of supplier performance metrics and ready to take action if needed?\n• How are “high risk” suppliers managed? Is it different than less risky suppliers?\n• Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n• Does the contractor have an adequate process in place to identify and ensure program requirements are properly flowed down to supplier/subcontractors?\n• Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n• Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n• What criteria are used to determine the capability/capacity of subcontractors and critical vendors/suppliers?\n• Identify the major/critical subcontractors/vendors for this program.\n• Are subcontractor/vendor delivery schedules compatible with production schedules?\n• How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?","Lessons Learned: \n• Look for a feedback loop to indicate that flow-down requirements are understood and verified. It is NOT sufficient for prime to communicate requirements without confirming understanding and acceptance.\n• Implementing a proactive and robust requirements flow-down process is a challenge. An undefined change control process is another area that can be a source of risk for the program. \n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area. \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality. Suppliers may not understand the application of their products into the overall system. The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
[230,4,"D - Materials","D.3 - Supply Chain Management","Has a survey for potential supply chain sources for the preferred materiel solution been completed?","Purpose: \nTo ensure: (1) Supply chain adequacy to support LRIP; (2) Adequate assessment of critical second and lower tier supply chain capability; (3) That proper processes and discipline are in place to ensure effective supplier requirements are being flowed down. Most prime contractors rely on their first tier suppliers to manage the lower tier. This methodology would be adequate if the prime could be certain that the first tier and second tier critical suppliers have the proper discipline to manage lower tier sub-contractors. Some primes perform Process Validation Audits (PVAs) on their first tier critical suppliers to ensure proper sub-tier management. Assessment of critical second and lower tier supply chain evidence assures the customer that the contractor has performed a thorough review of the critical manufacturing processes needed to produce the intended design at the supplier.","Additional Considerations:\n• Has an adequate assessment of the first tier supply chain been completed?\n• Is management aware of supplier performance metrics and ready to take action if needed?\n• How are “high risk” suppliers managed? Is it different than less risky suppliers?\n• Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n• Does the contractor have an adequate process in place to identify and ensure program requirements are properly flowed down to supplier/subcontractors?\n• Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n• Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n• What criteria are used to determine the capability/capacity of subcontractors and critical vendors/suppliers?\n• Identify the major/critical subcontractors/vendors for this program.\n• Are subcontractor/vendor delivery schedules compatible with production schedules?\n• How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?","Lessons Learned: \n• Look for a feedback loop to indicate that flow-down requirements are understood and verified. It is NOT sufficient for prime to communicate requirements without confirming understanding and acceptance.\n• Implementing a proactive and robust requirements flow-down process is a challenge. An undefined change control process is another area that can be a source of risk for the program. \n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area. \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality. Suppliers may not understand the application of their products into the overall system. The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
[231,4,"D - Materials","D.3 - Supply Chain Management","Have supply chain capability and capacity analyses been considered in the AoA?","Purpose: \nTo ensure: (1) Supply chain adequacy to support LRIP; (2) Adequate assessment of critical second and lower tier supply chain capability; (3) That proper processes and discipline are in place to ensure effective supplier requirements are being flowed down. Most prime contractors rely on their first tier suppliers to manage the lower tier. This methodology would be adequate if the prime could be certain that the first tier and second tier critical suppliers have the proper discipline to manage lower tier sub-contractors. Some primes perform Process Validation Audits (PVAs) on their first tier critical suppliers to ensure proper sub-tier management. Assessment of critical second and lower tier supply chain evidence assures the customer that the contractor has performed a thorough review of the critical manufacturing processes needed to produce the intended design at the supplier.","Additional Considerations:\n• Has an adequate assessment of the first tier supply chain been completed?\n• Is management aware of supplier performance metrics and ready to take action if needed?\n• How are “high risk” suppliers managed? Is it different than less risky suppliers?\n• Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n• Does the contractor have an adequate process in place to identify and ensure program requirements are properly flowed down to supplier/subcontractors?\n• Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n• Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n• What criteria are used to determine the capability/capacity of subcontractors and critical vendors/suppliers?\n• Identify the major/critical subcontractors/vendors for this program.\n• Are subcontractor/vendor delivery schedules compatible with production schedules?\n• How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?","Lessons Learned: \n• Look for a feedback loop to indicate that flow-down requirements are understood and verified. It is NOT sufficient for prime to communicate requirements without confirming understanding and acceptance.\n• Implementing a proactive and robust requirements flow-down process is a challenge. An undefined change control process is another area that can be a source of risk for the program. \n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area. \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality. Suppliers may not understand the application of their products into the overall system. The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
[232,5,"D - Materials","D.3 - Supply Chain Management","Have potential supply chain sources been identified and evaluated as able to support prototype build?","Purpose: \nTo ensure: (1) Supply chain adequacy to support LRIP; (2) Adequate assessment of critical second and lower tier supply chain capability; (3) That proper processes and discipline are in place to ensure effective supplier requirements are being flowed down. Most prime contractors rely on their first tier suppliers to manage the lower tier. This methodology would be adequate if the prime could be certain that the first tier and second tier critical suppliers have the proper discipline to manage lower tier sub-contractors. Some primes perform Process Validation Audits (PVAs) on their first tier critical suppliers to ensure proper sub-tier management. Assessment of critical second and lower tier supply chain evidence assures the customer that the contractor has performed a thorough review of the critical manufacturing processes needed to produce the intended design at the supplier.","Additional Considerations:\n• Has an adequate assessment of the first tier supply chain been completed?\n• Is management aware of supplier performance metrics and ready to take action if needed?\n• How are “high risk” suppliers managed? Is it different than less risky suppliers?\n• Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n• Does the contractor have an adequate process in place to identify and ensure program requirements are properly flowed down to supplier/subcontractors?\n• Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n• Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n• What criteria are used to determine the capability/capacity of subcontractors and critical vendors/suppliers?\n• Identify the major/critical subcontractors/vendors for this program.\n• Are subcontractor/vendor delivery schedules compatible with production schedules?\n• How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?","Lessons Learned: \n• Look for a feedback loop to indicate that flow-down requirements are understood and verified. It is NOT sufficient for prime to communicate requirements without confirming understanding and acceptance.\n• Implementing a proactive and robust requirements flow-down process is a challenge. An undefined change control process is another area that can be a source of risk for the program. \n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area. \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality. Suppliers may not understand the application of their products into the overall system. The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
[233,6,"D - Materials","D.3 - Supply Chain Management","Have requirements for the supply chain throughout the product lifecycle been updated?  "]
[234,6,"D - Materials","D.3 - Supply Chain Management","Has the critical suppliers list been updated?","Purpose: \nTo ensure: (1) Supply chain adequacy to support LRIP; (2) Adequate assessment of critical second and lower tier supply chain capability; (3) That proper processes and discipline are in place to ensure effective supplier requirements flow down. Most prime contractors rely on first tier suppliers to manage the lower tier. This methodology would be adequate if the prime is certain that the first tier and second tier critical suppliers have the proper discipline to manage lower tier sub-contractors. Some primes perform Process Validation Audits (PVAs) on first tier critical suppliers to ensure proper sub-tier management. Assessment of critical second and lower tier supply chain evidence assures the customer that the contractor has performed a thorough review of critical supplier manufacturing processes to produce the intended design.","Additional Considerations:\n• Is management aware of supplier performance metrics and ready to take action if needed?\n• Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n• Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n• Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n• Are subcontractor/vendor delivery schedules compatible with production schedules?\n• How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?","Lessons Learned: \n• Look for a feedback loop to indicate that flow-down requirements are understood and verified. It is NOT sufficient for prime to communicate requirements without confirming understanding and acceptance.\n• Implementing a proactive and robust requirements flow down process is a challenge. An undefined change control process is another area that can be a source of risk for the program. \n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area. \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality. Suppliers may not understand the application of their products into the overall system. The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
[235,6,"D - Materials","D.3 - Supply Chain Management","Are plans in place that include a supply chain with sufficient resources, capabilities, and capacities to support an awarding of an engineering and manufacturing (EMD) contract award?","Purpose: \nTo ensure: (1) Supply chain adequacy to support LRIP; (2) Adequate assessment of critical second and lower tier supply chain capability; (3) That proper processes and discipline are in place to ensure effective supplier requirements flow down. Most prime contractors rely on first tier suppliers to manage the lower tier. This methodology would be adequate if the prime is certain that the first tier and second tier critical suppliers have the proper discipline to manage lower tier sub-contractors. Some primes perform Process Validation Audits (PVAs) on first tier critical suppliers to ensure proper sub-tier management. Assessment of critical second and lower tier supply chain evidence assures the customer that the contractor has performed a thorough review of critical supplier manufacturing processes to produce the intended design.","Additional Considerations:\n• Is management aware of supplier performance metrics and ready to take action if needed?\n• Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n• Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n• Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n• Are subcontractor/vendor delivery schedules compatible with production schedules?\n• How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?","Lessons Learned: \n• Look for a feedback loop to indicate that flow-down requirements are understood and verified. It is NOT sufficient for prime to communicate requirements without confirming understanding and acceptance.\n• Implementing a proactive and robust requirements flow down process is a challenge. An undefined change control process is another area that can be a source of risk for the program. \n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area. \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality. Suppliers may not understand the application of their products into the overall system. The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
[236,7,"D - Materials","D.3 - Supply Chain Management","Is there an effective supply chain management process defined, documented and in place?","Purpose: \nTo ensure: (1) Supply chain adequacy to support LRIP; (2) Adequate assessment of critical second and lower tier supply chain capability; (3) That proper processes and discipline are in place to ensure effective supplier requirements flow down. Most prime contractors rely on first tier suppliers to manage the lower tier. This methodology would be adequate if the prime is certain that the first tier and second tier critical suppliers have the proper discipline to manage lower tier sub-contractors. Some primes perform Process Validation Audits (PVAs) on first tier critical suppliers to ensure proper sub-tier management. Assessment of critical second and lower tier supply chain evidence assures the customer that the contractor has performed a thorough review of critical supplier manufacturing processes to produce the intended design.","Additional Considerations:\n• Is management aware of supplier performance metrics and ready to take action if needed?\n• Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n• Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n• Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n• Are subcontractor/vendor delivery schedules compatible with production schedules?\n• How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?","Lessons Learned: \n• Look for a feedback loop to indicate that flow-down requirements are understood and verified. It is NOT sufficient for prime to communicate requirements without confirming understanding and acceptance.\n• Implementing a proactive and robust requirements flow down process is a challenge. An undefined change control process is another area that can be a source of risk for the program. \n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area. \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality. Suppliers may not understand the application of their products into the overall system. The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
[237,7,"D - Materials","D.3 - Supply Chain Management","Has a plan been developed for use of predictive indicators for risks in the supply chain?","Purpose: \nTo ensure: (1) Supply chain adequacy to support LRIP; (2) Adequate assessment of critical second and lower tier supply chain capability; (3) That proper processes and discipline are in place to ensure effective supplier requirements flow down. Most prime contractors rely on first tier suppliers to manage the lower tier. This methodology would be adequate if the prime is certain that the first tier and second tier critical suppliers have the proper discipline to manage lower tier sub-contractors. Some primes perform Process Validation Audits (PVAs) on first tier critical suppliers to ensure proper sub-tier management. Assessment of critical second and lower tier supply chain evidence assures the customer that the contractor has performed a thorough review of critical supplier manufacturing processes to produce the intended design.","Additional Considerations:\n• Is management aware of supplier performance metrics and ready to take action if needed?\n• Has the prime contractor flowed down the DPAS priority ratings to subcontractors and assured that the requirement has been passed on to lower tier vendors/suppliers? To other company divisions?\n• Does the prime contractor have a system that adequately controls subcontractors, vendors, and other company divisions?\n• Are there procedures for identifying material needs, lead times, and delivery schedules? How are these integrated into the production planning effort?\n• Are subcontractor/vendor delivery schedules compatible with production schedules?\n• How will you assure that subcontractors and vendors will meet the established schedules? Have you contracted with the vendors and subcontractors before, and what was their delivery performance?","Lessons Learned: \n• Look for a feedback loop to indicate that flow-down requirements are understood and verified. It is NOT sufficient for prime to communicate requirements without confirming understanding and acceptance.\n• Implementing a proactive and robust requirements flow down process is a challenge. An undefined change control process is another area that can be a source of risk for the program. \n• Following the trail of at least one engineering change through the entire company’s documented change control procedure is a valuable exercise when trying to ascertain a company’s robustness in this area. \n• Supplier change control has been a problem on some programs. Specifically, suppliers have made what they deem to be Class II changes, but these changes end up impacting the system performance or quality. Suppliers may not understand the application of their products into the overall system. The prime contractor needs to ensure supplier Class II changes are thoroughly reviewed to confirm they are truly Class II."]
[238,7,"D - Materials","D.3 - Supply Chain Management","Has an adequate assessment of the critical first tier supply chain been completed (i.e., capability, capacity, etc.)?"]
[239,8,"D - Materials","D.3 - Supply Chain Management","Has an assessment, including capability and capacity, of second and lower tier suppliers whose performance could seriously jeopardize the successful achievement of a program’s cost, schedule, technical, or supportability requirements been completed?","Purpose: \nTo demonstrate that the prime contractor has all the required elements of Supply Chain Management in place, and is actively managing the supply chain. Program requirements have been communicated and confirmed and supplier quality and delivery performance tracking metrics are in place. To the extent possible, long term agreements with critical suppliers are in place and are managed effectively. At this phase of the acquisition, the program has entered Full Rate Production (FRP). FRP entrance criteria have been accepted and passed. As many suppliers as is prudent should be signed to long-term agreements to provide price stability and availability of required materials. The Prime should have metrics in place to monitor the performance of suppliers along with systematic plan for dealing with underperforming suppliers. The supply base should have predictive indicators in place to call attention to potential problems in a proactive manner. The entire supply base should be known and stable with all suppliers aware of and able to support FRP quantities and delivery dates. Planning must be in place to improve the successful move to FRP and eliminate disruptions.","Additional Considerations:\n• Are long term agreements in place where practical?\n• Are the Prime's supplier management metrics (including thresholds and goals) in place and being used to manage risk?\n• How does the contractor control their suppliers' suppliers (i.e., lower tier suppliers)? How does the contractor evaluate their suppliers' subcontract management program?\n• Are predictive indicators in place to manage suppliers?\n• Does evidence of communications with suppliers and validation of requirements exist?\n• Does the contractor understand and maintain performance data of sub-tiers to quantify and manage risk associated with incoming material?","Lessons Learned: \n• It is critical to look for objective, not anecdotal, evidence to illustrate supplier compliance. Review the metrics for supplier management and demonstrate progression. \n• Ask to see evidence of sub-tier performance data and the problem solving process validation through corrective action reports from the sub-tier suppliers. \n• Capacity plans that address full rate production should focus on potential supplier bottlenecks. \n• Ask prime to show you how predictive indicators have helped them in proactively avoiding supplier problems before they become a disaster. \n• Ask when and how often supplier metrics are presented to senior leadership and look for examples of how suppliers are informed of their lackluster performance.\n• Risk reduction measures need to be taken for those sub-tier suppliers that have shown a history of poor performance to the defined metrics."]
[240,8,"D - Materials","D.3 - Supply Chain Management","Are robust flow-down processes for requirements in place?","Purpose: \nTo develop a basic understanding of what materials will be required for manufacturing and to identify material handling risk to support the analysis of alternatives in the MSA phase.\nThis product life-cycle level includes a basic identification of the manufacturing concepts and materials through paper studies. \n","Additional Considerations: \n• Is there a potential impact due to Environmental Safety and Occupational Health (ESOH) related laws (Clean Air Act, Clean Water Act, Toxic Substances Control Act, National Environmental Policy Act, System Safety, Hazardous Materials Management, etc.)?","Lessons Learned: \n• Special handling and ESOH considerations will impact all programs as they move forward. \n• Multiple laws will impact the program.\n• Multiple guides and instructions are available for program use as the program moves forward."]
[241,8,"D - Materials","D.3 - Supply Chain Management","Have the requirements flow-down processes been verified?","Purpose: \nTo develop an understanding of manufacturing feasibility and risk to support the analysis of alternatives in the MSA phase. In this stage of the product life-cycle level validation of manufacturing concepts begins through analytical or laboratory experiments. Materials and/or processes have been characterized for environmental concerns but further evaluation and demonstration is required. Experimental hardware models have been developed in a laboratory environment that may possess limited functionality.","Additional Considerations: \n• Is there a potential impact due to Environmental Safety and Occupational Health related laws (Clean Air Act, Clean Water Act, Toxic Substances Control Act, National Environmental Policy Act, System Safety, Hazardous Materials Management, etc.)?\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n","Lessons Learned: \n• Special handling and ESOH considerations will impact all programs as they move forward. \n• Multiple laws will impact the program.\n• Multiple guides and instructions are available for program use as the program moves forward.\n• At this point you probably have actual materials in your hands, if you were not impacted before you may be now. \n• Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices, make sure that these requirements are flowed down to the contractor so that you will be able to do your job."]
[242,8,"D - Materials","D.3 - Supply Chain Management","Has compliance with flowed-down requirements and any flowed-down changes been validated throughout the supply chain?","Purpose: \nTo develop an understanding of manufacturing feasibility and risk to support the analysis of alternatives in the MSA phase. In this stage of the product life-cycle level validation of manufacturing concepts begins through analytical or laboratory experiments. Materials and/or processes have been characterized for environmental concerns but further evaluation and demonstration is required. Experimental hardware models have been developed in a laboratory environment that may possess limited functionality.","Additional Considerations: \n• Is there a potential impact due to Environmental Safety and Occupational Health related laws (Clean Air Act, Clean Water Act, Toxic Substances Control Act, National Environmental Policy Act, System Safety, Hazardous Materials Management, etc.)?\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n","Lessons Learned: \n• Special handling and ESOH considerations will impact all programs as they move forward. \n• Multiple laws will impact the program.\n• Multiple guides and instructions are available for program use as the program moves forward.\n• At this point you probably have actual materials in your hands, if you were not impacted before you may be now. \n• Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices, make sure that these requirements are flowed down to the contractor so that you will be able to do your job."]
[243,8,"D - Materials","D.3 - Supply Chain Management","Has the plan been updated for use of predictive indicators during production?","Purpose: \nTo develop an understanding of manufacturing feasibility and risk to support the analysis of alternatives in the MSA phase. In this stage of the product life-cycle level validation of manufacturing concepts begins through analytical or laboratory experiments. Materials and/or processes have been characterized for environmental concerns but further evaluation and demonstration is required. Experimental hardware models have been developed in a laboratory environment that may possess limited functionality.","Additional Considerations: \n• Is there a potential impact due to Environmental Safety and Occupational Health related laws (Clean Air Act, Clean Water Act, Toxic Substances Control Act, National Environmental Policy Act, System Safety, Hazardous Materials Management, etc.)?\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n","Lessons Learned: \n• Special handling and ESOH considerations will impact all programs as they move forward. \n• Multiple laws will impact the program.\n• Multiple guides and instructions are available for program use as the program moves forward.\n• At this point you probably have actual materials in your hands, if you were not impacted before you may be now. \n• Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices, make sure that these requirements are flowed down to the contractor so that you will be able to do your job."]
[244,8,"D - Materials","D.3 - Supply Chain Management","Does the supply chain management system indicate that the supply chain is adequate, sufficient, and ready to support initial production (LRIP)?","Purpose: \nTo identify several potential material solutions and to assess the costs and risks associated with the preferred material approach. The exit criteria for this phase should include a Preliminary Hazards List (PHL) and Environmental, Safety, and Occupational Health (ESOH) criteria. Special handling requirements for hazardous materials and processes should have been identified. Special handling procedures applied in the lab. ESOH considerations for the next phase should have been identified to include initial ESOH planning. This phase looks at the capability to build the item in a laboratory environment. This includes technologies and manufacturing processes that might have ESOH considerations to include cost considerations. At this point you should have determined the following: (1) Mfg. Feasibility determined (can you build it?); (2) Mfg. Processes Identified (focus on processes with ESOH considerations); (3) Key Processes Identified and alternatives identified for those nasty processes; (4) Producibility Assessments Initiated\n","Additional Considerations: \n• Does the program have access to an ESOH professional?\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n","Lessons Learned: \n• Special handling and ESOH considerations will impact all programs as they move forward. \n• Multiple laws will impact the program (Clean Air Act, Clean Water Act, Toxic Substances Control Act, National Environmental Policy Act, System Safety, Hazardous Materials Management, etc.).\n• Multiple guides and instructions are available for program use as the program moves forward.\n• From here on you may be working with hazardous materials, but the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of HAZMAT. There are several requirements of program offices; make sure that these requirements are flowed down to the contractor so that you will be able to do your job."]
[245,9,"D - Materials","D.3 - Supply Chain Management","Are long term supplier agreements in place where practical?","Purpose: \nTo identify several potential material solutions and to assess the costs and risks associated with the preferred material approach. The exit criteria for this phase should include a Preliminary Hazards List (PHL) and Environmental, Safety, and Occupational Health (ESOH) criteria. Special handling requirements for hazardous materials and processes should have been identified. Special handling procedures applied in the lab. ESOH considerations for the next phase should have been identified to include initial ESOH planning. This phase looks at the capability to build the item in a laboratory environment. This includes technologies and manufacturing processes that might have ESOH considerations to include cost considerations. At this point you should have determined the following: (1) Mfg. Feasibility determined (can you build it?); (2) Mfg. Processes Identified (focus on processes with ESOH considerations); (3) Key Processes Identified and alternatives identified for those nasty processes; (4) Producibility Assessments Initiated\n","Additional Considerations: \n• Does the program have access to an ESOH professional?\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n","Lessons Learned: \n• Special handling and ESOH considerations will impact all programs as they move forward. \n• Multiple laws will impact the program (Clean Air Act, Clean Water Act, Toxic Substances Control Act, National Environmental Policy Act, System Safety, Hazardous Materials Management, etc.).\n• Multiple guides and instructions are available for program use as the program moves forward.\n• From here on you may be working with hazardous materials, but the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of HAZMAT. There are several requirements of program offices; make sure that these requirements are flowed down to the contractor so that you will be able to do your job."]
[246,9,"D - Materials","D.3 - Supply Chain Management","Are the supply chain management metrics, which include thresholds and goals in place?","Purpose: \nTo identify several potential material solutions and to assess the costs and risks associated with the preferred material approach. The exit criteria for this phase should include a Preliminary Hazards List (PHL) and Environmental, Safety, and Occupational Health (ESOH) criteria. Special handling requirements for hazardous materials and processes should have been identified. Special handling procedures applied in the lab. ESOH considerations for the next phase should have been identified to include initial ESOH planning. This phase looks at the capability to build the item in a laboratory environment. This includes technologies and manufacturing processes that might have ESOH considerations to include cost considerations. At this point you should have determined the following: (1) Mfg. Feasibility determined (can you build it?); (2) Mfg. Processes Identified (focus on processes with ESOH considerations); (3) Key Processes Identified and alternatives identified for those nasty processes; (4) Producibility Assessments Initiated\n","Additional Considerations: \n• Does the program have access to an ESOH professional?\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n",{"$f":"\"Lessons Learned: \n\"&G247","v":"Lessons Learned: \nAdditional Considerations: \n• Does the program have access to an ESOH professional?\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.\n"}]
[247,9,"D - Materials","D.3 - Supply Chain Management","Are the supply chain management metrics utilized to manage risks?"]
[248,9,"D - Materials","D.3 - Supply Chain Management","Is the use of predictive indicators to identify risks in the supply chain in place?"]
[249,9,"D - Materials","D.3 - Supply Chain Management","Does the supply chain management system indicate that the supply chain is adequate, sufficient, and ready to support rate production (FRP)?","Purpose: \nTo design and fabricate at the component level and determine cost, schedule and performance suitability. A secondary goal is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal. In the early stages of technology development effort the role of manufacturing is to “influence the design and plan for production.” The following are some design and planning concerns: (1) Environment, Safety, and Occupational Health (ESOH); (2) Programmatic Environmental, Safety, and Health Evaluation (PESHE); (3) National Environmental Policy Act (NEPA) Planning; (4) Human Systems Integration; (5) Systems Safety (Explosives, Laser, Noise, ManPrint, etc.); (6) Hazardous Material Management (HMM); (7) Pollution Prevention Planning.","\nAdditional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","\nLessons Learned: \n• NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program. Ensure time is scheduled for these activities. \n• You are most likely going to be working with HAZMAT. Plan for your involvement, but the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices; make sure that these requirements are flowed down to the contractor so that you will be able to do your job."]
[250,10,"D - Materials","D.3 - Supply Chain Management","Has the supply chain been proven in production and supports Full Rate Production (FRP)?","Purpose: \nTo design and fabricate at the component level and determine cost, schedule and performance suitability. A secondary goal is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal. In the early stages of technology development effort the role of manufacturing is to “influence the design and plan for production.” The following are some design and planning concerns: (1) Environment, Safety, and Occupational Health (ESOH); (2) Programmatic Environmental, Safety, and Health Evaluation (PESHE); (3) National Environmental Policy Act (NEPA) Planning; (4) Human Systems Integration; (5) Systems Safety (Explosives, Laser, Noise, ManPrint, etc.); (6) Hazardous Material Management (HMM); (7) Pollution Prevention Planning.","\nAdditional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","\nLessons Learned: \n• NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program. Ensure time is scheduled for these activities. \n• You are most likely going to be working with HAZMAT. Plan for your involvement, but the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices; make sure that these requirements are flowed down to the contractor so that you will be able to do your job."]
[251,1,"D - Materials","D.4 - Special Handling","Have hazardous materials been identified and safety procedures are in place? "]
[252,2,"D - Materials","D.4 - Special Handling","Has an initial evaluation of potential regulatory requirements and special handling concerns been completed?","Purpose: \nTo design and fabricate at the component level and determine cost, schedule and performance suitability. A secondary goal is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal. In the early stages of technology development effort the role of manufacturing is to “influence the design and plan for production.” The following are some design and planning concerns: (1) Environment, Safety, and Occupational Health (ESOH); (2) Programmatic Environmental, Safety, and Health Evaluation (PESHE); (3) National Environmental Policy Act (NEPA) Planning; (4) Human Systems Integration; (5) Systems Safety (Explosives, Laser, Noise, ManPrint, etc.); (6) Hazardous Material Management (HMM); (7) Pollution Prevention Planning.","\nAdditional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","\nLessons Learned: \n• NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program. Ensure time is scheduled for these activities. \n• You are most likely going to be working with HAZMAT. Plan for your involvement, but the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices; make sure that these requirements are flowed down to the contractor so that you will be able to do your job."]
[253,2,"D - Materials","D.4 - Special Handling","Have raw materials and components been assessed for special handling and potential regulatory requirements? "]
[254,3,"D - Materials","D.4 - Special Handling","Has ESH compliance risk been identified?","Purpose: \nTo design, fabricate, assemble and integrate at the sub-system level and determine cost, schedule and performance suitability. A secondary goal for this sub-thread is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal. In the early stages of technology development effort the role of manufacturing is to “influence the design and plan for production.” The following are some design and planning concerns: (1) Environment, Safety, and Occupational Health (ESOH); (2) Programmatic Environmental, Safety, and Health Evaluation (PESHE); (3) National Environmental Policy Act (NEPA) Planning; (4) Human Systems Integration; (5) Systems Safety (Explosives, Laser, Noise, ManPrint, etc.); (6) Hazardous Material Management (HMM); (7) Pollution Prevention Planning.\n","Additional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","\nLessons Learned: \n• NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program. Ensure time is scheduled for these activities.\n• You are most likely going to be working with HAZMAT. Plan for your involvement, but the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices; make sure that these requirements are flowed down to the contractor so that you will be able to do your job."]
[255,3,"D - Materials","D.4 - Special Handling","Has a list of hazardous materials been identified and alternatives evaluated?","Purpose: \nTo design, fabricate, assemble and integrate at the sub-system level and determine cost, schedule and performance suitability. A secondary goal for this sub-thread is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal. In the early stages of technology development effort the role of manufacturing is to “influence the design and plan for production.” The following are some design and planning concerns: (1) Environment, Safety, and Occupational Health (ESOH); (2) Programmatic Environmental, Safety, and Health Evaluation (PESHE); (3) National Environmental Policy Act (NEPA) Planning; (4) Human Systems Integration; (5) Systems Safety (Explosives, Laser, Noise, ManPrint, etc.); (6) Hazardous Material Management (HMM); (7) Pollution Prevention Planning.\n","Additional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","\nLessons Learned: \n• NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program. Ensure time is scheduled for these activities.\n• You are most likely going to be working with HAZMAT. Plan for your involvement, but the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices; make sure that these requirements are flowed down to the contractor so that you will be able to do your job."]
[256,3,"D - Materials","D.4 - Special Handling","Have special handling procedures been applied in the lab?"]
[257,3,"D - Materials","D.4 - Special Handling","Have special handling concerns been assessed?","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability. A secondary goal for this sub-thread is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal. In the early stages of technology development effort the role of manufacturing is to “influence the design and plan for production.” The following are some design and planning concerns: (1) Environment, Safety, and Occupational Health (ESOH); (2) Programmatic Environmental, Safety, and Health Evaluation (PESHE); (3) National Environmental Policy Act (NEPA) Planning; (4) Human Systems Integration; (5) Systems Safety (Explosives, Laser, Noise, ManPrint, etc.); (6) Hazardous Material Management (HMM); (7) Pollution Prevention Planning.","Additional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","Lessons Learned: \n• NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program. Ensure time is scheduled for these activities.\n• You are most likely going to be working with HAZMAT. Plan for your involvement, but the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices, make sure that these requirements are flowed down to the contractor so that you will be able to do your job.\n• Designing out hazardous material or material that needs special attention pays off. It costs eighty dollars to handle, treat and dispose of hazardous material for every one dollar spent to purchase the material!"]
[258,4,"D - Materials","D.4 - Special Handling","Has Environmental, Safety, and Health (ESH) compliance risks been mitigated in the lab environment?","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability. A secondary goal for this sub-thread is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal. In the early stages of technology development effort the role of manufacturing is to “influence the design and plan for production.” The following are some design and planning concerns: (1) Environment, Safety, and Occupational Health (ESOH); (2) Programmatic Environmental, Safety, and Health Evaluation (PESHE); (3) National Environmental Policy Act (NEPA) Planning; (4) Human Systems Integration; (5) Systems Safety (Explosives, Laser, Noise, ManPrint, etc.); (6) Hazardous Material Management (HMM); (7) Pollution Prevention Planning.","Additional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","Lessons Learned: \n• NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program. Ensure time is scheduled for these activities.\n• You are most likely going to be working with HAZMAT. Plan for your involvement, but the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices, make sure that these requirements are flowed down to the contractor so that you will be able to do your job.\n• Designing out hazardous material or material that needs special attention pays off. It costs eighty dollars to handle, treat and dispose of hazardous material for every one dollar spent to purchase the material!"]
[259,4,"D - Materials","D.4 - Special Handling","Has a list of hazardous materials been updated?"]
[260,4,"D - Materials","D.4 - Special Handling","Has the updated list of hazardous materials been assessed for alternative materials?"]
[261,4,"D - Materials","D.4 - Special Handling","Have special handling procedures been applied in a laboratory environment? ","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability. A secondary goal for this sub-thread is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal.","Additional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","Lessons Learned: \n• NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program. Ensure time is scheduled for these activities.\n• You are most likely going to be working with HAZMAT. Plan for your involvement, the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices, make sure that these requirements are flowed down to the contractor so that you will be able to do your job.\n• Designing out hazardous material or material that needs special attention pays off. It costs eighty dollars to handle, treat and dispose of hazardous material for every one dollar spent to purchase the material!"]
[262,4,"D - Materials","D.4 - Special Handling","Have disposal procedures been evaluated?","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability. A secondary goal for this sub-thread is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal.","Additional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","Lessons Learned: \n• NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program. Ensure time is scheduled for these activities.\n• You are most likely going to be working with HAZMAT. Plan for your involvement, the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices, make sure that these requirements are flowed down to the contractor so that you will be able to do your job.\n• Designing out hazardous material or material that needs special attention pays off. It costs eighty dollars to handle, treat and dispose of hazardous material for every one dollar spent to purchase the material!"]
[263,5,"D - Materials","D.4 - Special Handling","Have Environmental, Safety, and Health (ESH) requirements been applied in the production relevant environment?","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability. A secondary goal for this sub-thread is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal.","Additional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","Lessons Learned: \n• NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program. Ensure time is scheduled for these activities.\n• You are most likely going to be working with HAZMAT. Plan for your involvement, the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices, make sure that these requirements are flowed down to the contractor so that you will be able to do your job.\n• Designing out hazardous material or material that needs special attention pays off. It costs eighty dollars to handle, treat and dispose of hazardous material for every one dollar spent to purchase the material!"]
[264,5,"D - Materials","D.4 - Special Handling","Have special handling procedures been applied in the production relevant environment? ","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability. A secondary goal for this sub-thread is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal.","Additional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","Lessons Learned: \n• NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program. Ensure time is scheduled for these activities.\n• You are most likely going to be working with HAZMAT. Plan for your involvement, the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices, make sure that these requirements are flowed down to the contractor so that you will be able to do your job.\n• Designing out hazardous material or material that needs special attention pays off. It costs eighty dollars to handle, treat and dispose of hazardous material for every one dollar spent to purchase the material!"]
[265,5,"D - Materials","D.4 - Special Handling","Have special handling requirement gaps been identified?"]
[266,5,"D - Materials","D.4 - Special Handling","Have new special handling processes been demonstrated in lab environment?"]
[267,6,"D - Materials","D.4 - Special Handling","Have ESH requirements been addressed and documented?"]
[268,6,"D - Materials","D.4 - Special Handling","Have special handling procedures been demonstrated in a production relevant environment?","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability. A secondary goal for this sub-thread is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal.","Additional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","Lessons Learned: \n• Monitor compliance."]
[269,6,"D - Materials","D.4 - Special Handling","Are the plans to address the special handling requirement gaps, risks, and issues complete?","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability. A secondary goal for this sub-thread is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal.","Additional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","Lessons Learned: \n• Monitor compliance."]
[270,6,"D - Materials","D.4 - Special Handling","Has manufacturing been assessed for material storage and waste handling risks?","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability. A secondary goal for this sub-thread is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal.","Additional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","Lessons Learned: \n• Monitor compliance."]
[271,7,"D - Materials","D.4 - Special Handling","Has ESH compliance been demonstrated in a production representative environment?"]
[272,7,"D - Materials","D.4 - Special Handling","Have special handling procedures been applied in a production representative environment?","Purpose: \nTo design, fabricate, assemble and integrate at the system level and determine cost, schedule and performance suitability. A secondary goal for this sub-thread is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal.","Additional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","Lessons Learned: \n• Continue to monitor risks and continuously improve."]
[273,7,"D - Materials","D.4 - Special Handling","Have special handling procedures been developed and annotated on the work instructions to be used for the pilot line?"]
[274,7,"D - Materials","D.4 - Special Handling","Is there a hazardous material storage and disposal plan in place for the pilot line?"]
[275,8,"D - Materials","D.4 - Special Handling","Has ESH compliance been demonstrated during pilot line production?","Purpose: \nTo begin addressing the need for models to simulate the manufacturing of the projects and to use this information to understand the risk in pursuing this approach. To allow decision makers insight into the risk and start exploring mitigation options to address those risks identified. Modeling and simulations in these early phases can provide needed in-sight into the risk of the products being considered and developed.","Additional Considerations: \n• None","Lessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address what models and simulation tools we will use to assess the manufacturing operation earlier in the acquisition process."]
[276,8,"D - Materials","D.4 - Special Handling","Have special handling procedures been applied in a pilot line environment?  ","Purpose: \nTo begin identifying manufacturing risk for the alternative being considered, and to use this information to understand the manufacturing risk to help manage the transition. To see some high level process charts to support this effort and for new processes have some data available based on a minimum of experimental data. The project needs identify manufacturing risk to produce each alternative being considered. At the end of this Pre-MSA phase the best alternatives will be selected to address user requirements to proceed into MSA.","Additional Considerations: \n• Do process flow charts accurately capture all critical process operations?\n","Lessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address what models and simulation tools we will use to assess the manufacturing operation. \n• To aid in this assessment, the program should address the manufacturing approach being considered using as a minimum a high-level process flow chart and identify the risk associated with using this approach."]
[277,8,"D - Materials","D.4 - Special Handling","Have special handling procedures been demonstrated in Engineering and Manufacturing Development (EMD) or technology insertion programs?    ","Purpose: \no understand the modeling/simulation approach that will be used to identify the risk drivers in manufacturing for alternatives being considered, and to use this information to understand the insight in the risk in selecting the best alternative. The program needs to project and understand the tools used to assess manufacturing risk to produce proposed alternatives. At the end of this phase the best alternatives will be selected to address user requirements to proceed into TD.\n","Additional Considerations: \n• Do process flow charts accurately capture all critical process operations?\n","Lessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must start addressing what models and simulation tools we will use to assess the manufacturing operation earlier in the acquisition process. \n• Value Stream Maps are a useful tool to understand manufacturing constraints."]
[278,8,"D - Materials","D.4 - Special Handling","Have the special handling risks and issues been managed for Low Rate Initial Production (LRIP)?","Purpose: \nTo design and fabricate at the component level and determine cost, schedule and performance suitability. A secondary goal is not to have to address ESOH or special handling considerations. The priority approach or strategy should be in the following order: (1) Source Reduction (design out); (2) Recycle (plan for in design and production planning); (3) Treatment; (4) Disposal. In the early stages of technology development effort the role of manufacturing is to “influence the design and plan for production.” The following are some design and planning concerns: (1) Environment, Safety, and Occupational Health (ESOH); (2) Programmatic Environmental, Safety, and Health Evaluation (PESHE); (3) National Environmental Policy Act (NEPA) Planning; (4) Human Systems Integration; (5) Systems Safety (Explosives, Laser, Noise, ManPrint, etc.); (6) Hazardous Material Management (HMM); (7) Pollution Prevention Planning.","\nAdditional Considerations:\n• Special handling requirements may also include product movement in the factory or shipping between suppliers and the prime due to size and/or weight, cleanliness, and fragility.","\nLessons Learned: \n• NEPA planning and the need for community awareness and time to assess impacts could add 6 months or more to a program. Ensure time is scheduled for these activities. \n• You are most likely going to be working with HAZMAT. Plan for your involvement, but the best option is to design hazardous materials out of your program. Know the laws, DOD policies and regulations related to the handling of hazardous materials. There are several requirements of program offices; make sure that these requirements are flowed down to the contractor so that you will be able to do your job."]
[279,8,"D - Materials","D.4 - Special Handling","Do all work instructions contain required special handling provisions?","Purpose: \nTo obtain more fidelity in the models/simulations at the sub-system or system level by updating previous component models using data from building and testing prototypes in a relevant environment. The program must obtain more realism in assessing manufacturing risk/constraints by using actual build data or highly realistic simulation models. This will ensure an adequate assessment of manufacturing risk posed by the final design. At the end of this phase the best alternative will be selected to proceed into EMD.\n","Additional Considerations: \n• The models/simulations at this stage should be at the subsystem level and address specific manufacturing areas to identify constraints for each prototype being built (e.g., materials, labor, equipment, tooling/Special Test Equipment (STE), setup, yield/scrap/ rework, Work In Progress (WIP), and capability/capacity). \n• Are supplier processes included in the overall flow, including lead times for critical items?","Lessons Learned: \n• Any simulation models need to incorporate actual build data where available, continually be updated to reflect changes, and be representative of the whole system."]
[280,8,"D - Materials","D.4 - Special Handling","Have hazardous material storage and disposal plans for LRIP been evaluated?"]
[281,8,"D - Materials","D.4 - Special Handling","Are hazardous material storage and disposal plans in place for LRIP?","Purpose: \nTo use demonstrated results from a production representative environment to verify accuracy of models/simulations and use this information to identify risk/constraints and improvement opportunities. The program needs verify the realism of the models/simulations by using actual build data from a production representative environment.","Additional Considerations: \n• The models/simulations at this stage should be at the system level and address specific manufacturing areas to identify constraints for each prototype being built (e.g., materials, labor, equipment, tooling/Special Test Equipment (STE), setup, yield/scrap/ rework, Work In Progress (WIP), and capability/capacity).","Lessons Learned: \n• Continually update models/simulation tools as more data becomes available. \n• Accurate models and simulations will identify constraints and improvement opportunities early and provide time to work the issues that arise before transition to production."]
[282,9,"D - Materials","D.4 - Special Handling","Has ESH compliance been demonstrated in LRIP?"]
[283,9,"D - Materials","D.4 - Special Handling","Have special handling procedures been demonstrated in a Low Rate Initial Production (LRIP) environment? ","Purpose: \nTo use demonstrated results from a pilot line environment to verify accuracy of models/ simulations and to use this information to identify risks/constraints and improvement opportunities. To assess whether you can meet LRIP requirements. The program manager must verify the realism of models/simulations by using actual build data from a pilot line to ensure confidence in the ability to identify manufacturing risk/constraints and improvement opportunities.","Additional Considerations: \n• Models and simulations are only useful if they reflect actual build data prior to entering LRIP.","Lessons Learned: \n• Continually update models and simulation tools as more build data becomes available, and use this information to identify risk areas as you transition into production."]
[284,9,"D - Materials","D.4 - Special Handling","Have hazardous material storage and disposal procedures been demonstrated during Low Rate Initial Production (LRIP)?","Purpose: \nTo use demonstrated results from a pilot line environment to verify accuracy of models/ simulations and to use this information to identify risks/constraints and improvement opportunities. To assess whether you can meet LRIP requirements. The program manager must verify the realism of models/simulations by using actual build data from a pilot line to ensure confidence in the ability to identify manufacturing risk/constraints and improvement opportunities.","Additional Considerations: \n• Models and simulations are only useful if they reflect actual build data prior to entering LRIP.","Lessons Learned: \n• Continually update models and simulation tools as more build data becomes available, and use this information to identify risk areas as you transition into production."]
[285,9,"D - Materials","D.4 - Special Handling","Are risks and issues from special handling and hazardous materials storage and disposal managed for Full Rate Production (FRP)?"]
[286,10,"D - Materials","D.4 - Special Handling","Has ESH compliance been demonstrated in FRP? ","Purpose: \nTo use build data from LRIP to validate models/simulations and to use this information to identify risk/constraints and improvement opportunities. To use these tools to assess if you can meet your FRP program requirements The program must constantly verify realism in the cost and process models by using demonstrated data from a LRIP environment to ensure confidence in achieving program requirements.","Additional Considerations: \n• None","Lessons Learned: \n• Continually update models with actual build data and use those tools to assess future risk."]
[287,10,"D - Materials","D.4 - Special Handling","Have the special handling procedures been effectively implemented in FRP?  ","Purpose: \nTo use build data from LRIP to validate models/simulations and to use this information to identify risk/constraints and improvement opportunities. To use these tools to assess if you can meet your FRP program requirements The program must constantly verify realism in the cost and process models by using demonstrated data from a LRIP environment to ensure confidence in achieving program requirements.","Additional Considerations: \n• None","Lessons Learned: \n• Continually update models with actual build data and use those tools to assess future risk."]
[288,10,"D - Materials","D.4 - Special Handling","Have the hazardous material storage and disposal procedures been effectively implemented in FRP?  "]
[289,1,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have modeling and simulation approaches/tools been identified to support manufacturing and quality activities?","Purpose: \nTo use demonstrated results from FRP to verify you can meet FRP program requirements. To use these tools to help manage the FRP efforts. The program build data should support program requirements.","Additional Considerations: \n• None","\nLessons Learned: \n• Continually update models with actual build data and use those tools to assess future risk."]
[290,2,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have modeling and simulation in development been initiated?","Purpose: \nTo use demonstrated results from FRP to verify you can meet FRP program requirements. To use these tools to help manage the FRP efforts. The program build data should support program requirements.","Additional Considerations: \n• None","\nLessons Learned: \n• Continually update models with actual build data and use those tools to assess future risk."]
[291,3,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have manufacturing and quality gaps for system concepts been identified using modeling and simulation?","Purpose: \nTo begin identifying manufacturing requirements to build the items (e.g., material and/or process approaches), and to use this information to understand risk in pursuing this approach. To allow decision makers insight into risk and start exploring mitigation options to address those risks. The R&D community needs a basic understanding on the manufacturing requirements and the risk and implications of the approach being considered.","Additional Considerations: \n• None","Lessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address manufacturing risk/issues earlier in the R&D process."]
[292,4,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have production modeling and simulation tools been utilized to define manufacturing and quality requirements for the preferred materiel solution?","Purpose: \nTo begin identifying and understanding the capability of critical manufacturing processes of your product through experimentation, and to use this information to understand the risk in pursuing this approach. To allow decision makers insight into the manufacturing risk and start exploring mitigation options to address those risks identified. Assessing the maturity of the manufacturing processes in developmental items mandates that the S&T and acquisition communities work together to assess manufacturing risk for every alternative being considered.","Additional Considerations: \n• It is especially critical that the cost goals reflect manufacturing cost considerations and capabilities.\n","Lessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address manufacturing risk/issues earlier in the R&D process."]
[293,4,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have modeling and simulation results been considered in the AoA?","Purpose: \nTo begin identifying and understanding the capability of critical manufacturing processes of your product through experimentation, and to use this information to understand the risk in pursuing this approach. To allow decision makers insight into the manufacturing risk and start exploring mitigation options to address those risks identified. Assessing the maturity of the manufacturing processes in developmental items mandates that the S&T and acquisition communities work together to assess manufacturing risk for every alternative being considered.","Additional Considerations: \n• It is especially critical that the cost goals reflect manufacturing cost considerations and capabilities.\n","Lessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address manufacturing risk/issues earlier in the R&D process."]
[294,5,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have the initial modeling & simulation (product or process) been developed at the component level and used to determine constraints?","Purpose: \nTo begin assessing the current demonstrated capability of critical manufacturing processes and to understand the issues/risks for supporting the AoA. To use this information to understand the manufacturing risk in selecting the alternatives to proceed to the TD phase. The program manager must understand the manufacturing risk of each design alternative and include that risk in program planning efforts. A critical step is to perform a capability assessment of critical processes and quantify the risk for each alternative being considered. At the end of this phase the best available alternatives will be selected to proceed into TD. \n","Additional Considerations: \n• What are the critical processes that need attention at this stage of the product life-cycle?",{"$f":"\"Lessons Learned: \n\"&G295","v":"Lessons Learned: \nAdditional Considerations: \n• What are the critical processes that need attention at this stage of the product life-cycle?"}]
[295,6,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have the initial modeling & simulation been developed at the sub-system or system level?","Purpose: \nTo obtain more fidelity assessing process capability by using data from building prototypes and similar processes to assess whether target costs are achievable. Data generated from this assessment will be used to make design choices, make/buy decisions, and capacity, process capability, sources, quality, key characteristics, yield/rate, and variability assessments. The program manager must continue to assess current process capability versus what is needed to meet program requirements. Using actual build data can ensure adequate assessment of manufacturing risk transitioning to production. At the end of this phase you will select the best alternatives to proceed into EMD.","\nAdditional Considerations: \n• You should understand risk that each selected alternative presents--especially for critical processes.","Lessons Learned: \n• The need for realism in the process capability assessment requires using actual hardware build data as soon as possible."]
[296,6,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have the initial modeling & simulations been used to determine system constraints?","Purpose: \nTo obtain more fidelity assessing process capability by using data from building prototypes and similar processes to assess whether target costs are achievable. Data generated from this assessment will be used to make design choices, make/buy decisions, and capacity, process capability, sources, quality, key characteristics, yield/rate, and variability assessments. The program manager must continue to assess current process capability versus what is needed to meet program requirements. Using actual build data can ensure adequate assessment of manufacturing risk transitioning to production. At the end of this phase you will select the best alternatives to proceed into EMD.","\nAdditional Considerations: \n• You should understand risk that each selected alternative presents--especially for critical processes.","Lessons Learned: \n• The need for realism in the process capability assessment requires using actual hardware build data as soon as possible."]
[297,7,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have the modeling & simulations been used to determine system constraints?","Purpose: \nTo demonstrate process capability in a production relevant environment. To obtain more fidelity in assessing process capability by using data from building prototypes of from similar production processes to determine whether program objectives are achievable. Data generated from this assessment will be used to make design choices, make/buy decisions, and capacity, process capability, sources, quality, key characteristics, yield/rate, and variability assessments. The program is required to produce in a production relevant environment and the data will compare existing process capability versus program requirements. Using actual prototype build data in performing these assessments will ensure adequate assessment of risk transitioning to production. At the end of this phase you will select the best alternatives available to proceed into EMD.","Additional Considerations: \n• You should understand risk that each selected alternative presents--especially for critical processes.","\n\nLessons Learned: \n• The need for realism in the process capability assessment requires using actual hardware build data as soon as possible."]
[298,7,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have the modeling & simulations been used to identify improvement opportunities?","Purpose: \nTo demonstrate process capability in a production relevant environment. To obtain more fidelity in assessing process capability by using data from building prototypes of from similar production processes to determine whether program objectives are achievable. Data generated from this assessment will be used to make design choices, make/buy decisions, and capacity, process capability, sources, quality, key characteristics, yield/rate, and variability assessments. The program is required to produce in a production relevant environment and the data will compare existing process capability versus program requirements. Using actual prototype build data in performing these assessments will ensure adequate assessment of risk transitioning to production. At the end of this phase you will select the best alternatives available to proceed into EMD.","Additional Considerations: \n• You should understand risk that each selected alternative presents--especially for critical processes.","\n\nLessons Learned: \n• The need for realism in the process capability assessment requires using actual hardware build data as soon as possible."]
[299,8,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have the modeling & simulations been verified by the pilot line build?"]
[300,8,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have the results of pilot line modeling & simulations been used to improve process? ","Purpose: \nTo demonstrate process capability in a production representative environment. To obtain more fidelity in assessing process capability by using data from building EMD units and similar processes in production to determine whether program objectives are achievable. Data generated from this assessment will be used to make design choices, make/buy decisions, and capacity, process capability, sources, quality, key characteristics, yield/rate, and variability assessments. At this phase the data must ensure process capability to meet program requirements. It is essential to use real data performing these assessments to ensure adequate risk assessment of transitioning to production. At the end of this phase you will transition into LRIP.","Additional Considerations: \n• What impact will design changes have on process capability if different or changed processes are required?\n• Taguchi Design of Experiments (DOE) is an effective approach to control process variability.\n","Lessons Learned: \n• Continually assess demonstrated process capability as more data becomes available, and compare it to production requirements."]
[301,8,"E - Process Capability & Control","E.1 - Modeling & Simulation","Do the results of pilot line modeling & simulations demonstrate that Low Rate Initial Production (LRIP) requirements can be met?","Purpose: \nTo demonstrate process capability in a production representative environment. To obtain more fidelity in assessing process capability by using data from building EMD units and similar processes in production to determine whether program objectives are achievable. Data generated from this assessment will be used to make design choices, make/buy decisions, and capacity, process capability, sources, quality, key characteristics, yield/rate, and variability assessments. At this phase the data must ensure process capability to meet program requirements. It is essential to use real data performing these assessments to ensure adequate risk assessment of transitioning to production. At the end of this phase you will transition into LRIP.","Additional Considerations: \n• What impact will design changes have on process capability if different or changed processes are required?\n• Taguchi Design of Experiments (DOE) is an effective approach to control process variability.\n","Lessons Learned: \n• Continually assess demonstrated process capability as more data becomes available, and compare it to production requirements."]
[302,9,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have modeling & simulations been verified by Low Rate Initial Production (LRIP) build?","Purpose: \nTo demonstrate process capability in a production representative environment. To obtain more fidelity in assessing process capability by using data from building EMD units and similar processes in production to determine whether program objectives are achievable. Data generated from this assessment will be used to make design choices, make/buy decisions, and capacity, process capability, sources, quality, key characteristics, yield/rate, and variability assessments. At this phase the data must ensure process capability to meet program requirements. It is essential to use real data performing these assessments to ensure adequate risk assessment of transitioning to production. At the end of this phase you will transition into LRIP.","Additional Considerations: \n• What impact will design changes have on process capability if different or changed processes are required?\n• Taguchi Design of Experiments (DOE) is an effective approach to control process variability.\n","Lessons Learned: \n• Continually assess demonstrated process capability as more data becomes available, and compare it to production requirements."]
[303,9,"E - Process Capability & Control","E.1 - Modeling & Simulation","Are modeling & simulations used to assist in the management of Low Rate Initial Production (LRIP)?","Purpose: \nTo demonstrate that manufacturing process capabilities are proven to be adequate in a pilot line production environment, to ensure ability to achieve Low Rate Initial Production objectives.\nA Pilot line demonstration provides verifiable process capability/ performance results to compare to LRIP requirements. Some of the data includes information to determine whether tolerances can be achieved for all Key Characteristics. It is essential to use actual pilot line build data to ensure manufacturing risk of transitioning into production has been adequately assessed.","Additional Considerations: \n• Taguchi Design of Experiments (DOE) is an effective approach to control process variability.","Lessons Learned: \n1. One of the important steps for achieving this requirement is to work with the contractor/ supplier on what represents a pilot production line for your programs. This effort should be done as soon as practical and if at all possible, before the award of the contract for EMD. \n2. This difficult task will require SMEs to help define the requirements that best support your unique program requirements. \n3. What future rate and performance requirements and what critical processes need to be monitored on the Pilot line to ensure LRIP cost, schedule, and performance objectives will be met. \n4. Focusing in on process capability measures via Statistical Process Control (SPC) may not be possible due to small lot sizes; therefore, look at other key performance output measures to determine the process capability (e.g., first pass yield, scrap/rework, cycle time, etc.)."]
[304,9,"E - Process Capability & Control","E.1 - Modeling & Simulation","Do the results of Low Rate Initial Production (LRIP) modeling & simulations demonstrate that Full Rate Production (FRP) requirements can be met?  ","Purpose: \nTo demonstrate that manufacturing process capabilities are proven to be adequate in a pilot line production environment, to ensure ability to achieve Low Rate Initial Production objectives.\nA Pilot line demonstration provides verifiable process capability/ performance results to compare to LRIP requirements. Some of the data includes information to determine whether tolerances can be achieved for all Key Characteristics. It is essential to use actual pilot line build data to ensure manufacturing risk of transitioning into production has been adequately assessed.","Additional Considerations: \n• Taguchi Design of Experiments (DOE) is an effective approach to control process variability.","Lessons Learned: \n1. One of the important steps for achieving this requirement is to work with the contractor/ supplier on what represents a pilot production line for your programs. This effort should be done as soon as practical and if at all possible, before the award of the contract for EMD. \n2. This difficult task will require SMEs to help define the requirements that best support your unique program requirements. \n3. What future rate and performance requirements and what critical processes need to be monitored on the Pilot line to ensure LRIP cost, schedule, and performance objectives will be met. \n4. Focusing in on process capability measures via Statistical Process Control (SPC) may not be possible due to small lot sizes; therefore, look at other key performance output measures to determine the process capability (e.g., first pass yield, scrap/rework, cycle time, etc.)."]
[305,10,"E - Process Capability & Control","E.1 - Modeling & Simulation","Have modeling & simulations been verified by Full Rate Production (FRP) build?","Purpose: \nTo demonstrate that manufacturing process capabilities are proven to be adequate in a pilot line production environment, to ensure ability to achieve Low Rate Initial Production objectives.\nA Pilot line demonstration provides verifiable process capability/ performance results to compare to LRIP requirements. Some of the data includes information to determine whether tolerances can be achieved for all Key Characteristics. It is essential to use actual pilot line build data to ensure manufacturing risk of transitioning into production has been adequately assessed.","Additional Considerations: \n• Taguchi Design of Experiments (DOE) is an effective approach to control process variability.","Lessons Learned: \n1. One of the important steps for achieving this requirement is to work with the contractor/ supplier on what represents a pilot production line for your programs. This effort should be done as soon as practical and if at all possible, before the award of the contract for EMD. \n2. This difficult task will require SMEs to help define the requirements that best support your unique program requirements. \n3. What future rate and performance requirements and what critical processes need to be monitored on the Pilot line to ensure LRIP cost, schedule, and performance objectives will be met. \n4. Focusing in on process capability measures via Statistical Process Control (SPC) may not be possible due to small lot sizes; therefore, look at other key performance output measures to determine the process capability (e.g., first pass yield, scrap/rework, cycle time, etc.)."]
[306,10,"E - Process Capability & Control","E.1 - Modeling & Simulation","Are modeling & simulations used to assist in the management of Full Rate Production (FRP)?","Purpose: \nTo use demonstrated results from a LRIP line to verify processes are stable, adequately controlled, and capable. To use build data to analyze the process capability to identify areas for improvement. The program needs to constantly assess the process capability by using actual build data from a LRIP environment to ensure confidence in meeting program objectives. This early production data will set the foundation for future production cost and performance. \n","Additional Considerations: \n• Taguchi Design of Experiments (DOE) is an effective approach to control process variability.","Lessons Learned: \n• Continually update process capability assessments as more data becomes available and use this data to determine the likelihood of achieving program objectives."]
[307,1,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have hypotheses been developed regarding cause-effect relationships between process variables and process stability and repeatability?","Purpose: \nTo use demonstrated results from a LRIP line to verify processes are stable, adequately controlled, and capable. To use build data to analyze the process capability to identify areas for improvement. The program needs to constantly assess the process capability by using actual build data from a LRIP environment to ensure confidence in meeting program objectives. This early production data will set the foundation for future production cost and performance. \n","Additional Considerations: \n• Taguchi Design of Experiments (DOE) is an effective approach to control process variability.","Lessons Learned: \n• Continually update process capability assessments as more data becomes available and use this data to determine the likelihood of achieving program objectives."]
[308,2,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have studies been performed to test hypotheses regarding cause-effect relationships?"]
[309,2,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have initial process approaches been identified?"]
[310,3,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have cause-effect relationships between process control variables and process stability and repeatability been validated through laboratory experiments?","Purpose: \nTo demonstrate process capability to meet FRP objectives. The program actual build data must support program objectives.","Additional Considerations: \n• None","Lessons Learned: \n• Continually update process capability assessments as more data becomes available and use this data to determine the likelihood of continually meeting program objectives."]
[311,3,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have critical process control variables been identified?"]
[312,4,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Has the maturity of critical processes for the preferred materiel solution been assessed?","Purpose: \nTo a get a baseline on the yields and rates demonstrated in R&D. To effectively transition R&D projects one should address the manufacturing maturity of the prototypes being developed. The actual yields achieved on these projects will provide both the S&T and acquisition community a good indicator of the process maturity and the risk of proceeding.","Additional Considerations: \n• None","Lessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address manufacturing maturity and yield issues earlier in the R&D process."]
[313,4,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have process capability requirements and improvement plans been developed and documented in the SEP? ","Purpose: \nTo understand the yields and rates achieved on the R&D project or on similar process used by this contractor to assess the manufacturing risk of proceeding to Milestone A and to reflect that risk in the AoA. The yields and rates that have been demonstrated to date, either through R&D or a similar process, need to be compared with program requirements. This takes some judgment and should be done carefully; however, it is an excellent indicator of program risk.\n","Additional Considerations: \n• None\n","Lessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address manufacturing risk/issues earlier in the R&D process.\n• The yields being demonstrated versus what are required is a best indicator to provide the current maturity and risk of proceeding forward.\n• If yields are below 60% on critical processes at this stage, you need to monitor this process very closely."]
[314,5,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Has process maturity been assessed on similar processes currently in production?","Purpose: \nTo understand the yields and rates achieved on the R&D project or on similar process used by this contractor to assess the manufacturing risk of proceeding to Milestone A and to reflect that risk in the AoA. The yields and rates that have been demonstrated to date, either through R&D or a similar process, need to be compared with program requirements. This takes some judgment and should be done carefully; however, it is an excellent indicator of program risk.\n","Additional Considerations: \n• None\n","Lessons Learned: \n• Historically we have not adequately addressed manufacturing risk in this phase of technology development, so we must address manufacturing risk/issues earlier in the R&D process.\n• The yields being demonstrated versus what are required is a best indicator to provide the current maturity and risk of proceeding forward.\n• If yields are below 60% on critical processes at this stage, you need to monitor this process very closely."]
[315,5,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have process capability requirements been identified for pilot line, Low Rate Initial Production (LRIP) and Full Rate Production (FRP)?","Purpose: \nTo understand the current yields and rates and develop targets for growth of these factors at key milestones in the development cycle (e.g., pilot line, LRP, and FRP). To ensure improvements plans have been developed and are being implemented to achieve these growth targets. The program manager must assess the current yields and rates against program requirements and implement plans to achieve required targets. Further, if there is no other option, the progress in improving your yields and rates may be the key factor in determining program success.","\nAdditional Considerations: \n• It is important at this phase to develop process improvement plans where design trade-offs and process selection can be made easier to achieve program requirements.","Lessons Learned: \n• The yields being demonstrated versus what are required is a best indicator to provide the current maturity and risk of proceeding forward.\n• If yields are below 60% on critical processes at this stage, you need to monitor this process very closely."]
[316,6,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have the key manufacturing processes been demonstrated in a production relevant environment?","Purpose: \nTo understand the current yields and rates and develop targets for growth of these factors at key milestones in the development cycle (e.g., pilot line, LRP, and FRP). To ensure improvements plans have been developed and are being implemented to achieve these growth targets. The program manager must assess the current yields and rates against program requirements and implement plans to achieve required targets. Further, if there is no other option, the progress in improving your yields and rates may be the key factor in determining program success.","\nAdditional Considerations: \n• It is important at this phase to develop process improvement plans where design trade-offs and process selection can be made easier to achieve program requirements.","Lessons Learned: \n• The yields being demonstrated versus what are required is a best indicator to provide the current maturity and risk of proceeding forward.\n• If yields are below 60% on critical processes at this stage, you need to monitor this process very closely."]
[317,6,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Has collection or estimation of process capability data from prototype builds been initiated?","Purpose: \nTo understand the current yields and rates and develop targets for growth of these factors at key milestones in the development cycle (e.g., pilot line, LRP, and FRP). To ensure improvements plans have been developed and are being implemented to achieve these growth targets. The program manager must assess the current yields and rates against program requirements and implement plans to achieve required targets. Further, if there is no other option, the progress in improving your yields and rates may be the key factor in determining program success.","\nAdditional Considerations: \n• It is important at this phase to develop process improvement plans where design trade-offs and process selection can be made easier to achieve program requirements.","Lessons Learned: \n• The yields being demonstrated versus what are required is a best indicator to provide the current maturity and risk of proceeding forward.\n• If yields are below 60% on critical processes at this stage, you need to monitor this process very closely."]
[318,6,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Has refinement of process capability requirements from prototype builds been initiated?","Purpose: \nTo assess the yields and rates achieved building prototypes in a production relevant environment, compare them to targets, and determine the manufacturing risk of meeting future requirements. To use this data to make the appropriate mitigation/improvement plans to achieve future requirements. The program manager must understand current yields and rates and compare those to requirements. If this is not addressed before EMD it will become a major problem.","Additional Considerations: \n• Exercise caution, because even if you meet yield requirements in pre-EMD, the risk could still be high for production quantities.","\nLessons Learned: \n• The yields being demonstrated versus what are required is a best indicator to provide the current maturity and risk of proceeding forward.\n• If yields are below 60% on critical processes at this stage, you need to monitor this process very closely.\n• This is a good to time to assess design requirements versus process capability to see if opportunities exist to improve yields and rates without compromising performance requirements."]
[319,7,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have the manufacturing processes been successfully demonstrated in a production representative environment?","Purpose: \nTo assess the yields and rates achieved building prototypes in a production relevant environment, compare them to targets, and determine the manufacturing risk of meeting future requirements. To use this data to make the appropriate mitigation/improvement plans to achieve future requirements. The program manager must understand current yields and rates and compare those to requirements. If this is not addressed before EMD it will become a major problem.","Additional Considerations: \n• Exercise caution, because even if you meet yield requirements in pre-EMD, the risk could still be high for production quantities.","\nLessons Learned: \n• The yields being demonstrated versus what are required is a best indicator to provide the current maturity and risk of proceeding forward.\n• If yields are below 60% on critical processes at this stage, you need to monitor this process very closely.\n• This is a good to time to assess design requirements versus process capability to see if opportunities exist to improve yields and rates without compromising performance requirements."]
[320,7,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Is collection and/or estimation of process capability data (i.e., from lower level sub-systems, items, and components) ongoing?","Purpose: \nTo use results from a production representative environment to demonstrated that you can achieve pilot line requirements for yields and rates. To use your data for updating your mitigation/improvement plans required to achieve targeted yields and rate requirements. The program needs to verify the likelihood of achieving production requirements from demonstrated data built in a production representative environment. The program needs data to make updates to improvement plans.","Additional Considerations: \n• Exercise caution, because even if you meet yield requirements in EMD, the risk could still be high for full-rate production quantities.","Lessons Learned: \n• Continually update yield and rate targets as more data becomes available and use the data to make informed program decisions and process improvements. \n• Not achieving targets by this time will demonstrate the need to closely monitor this effort and identify mitigations to minimize the impact of not achieving your targets."]
[321,7,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Is refinement of requirements at all levels for process capabilities ongoing?","Purpose: \nTo use results from a production representative environment to demonstrated that you can achieve pilot line requirements for yields and rates. To use your data for updating your mitigation/improvement plans required to achieve targeted yields and rate requirements. The program needs to verify the likelihood of achieving production requirements from demonstrated data built in a production representative environment. The program needs data to make updates to improvement plans.","Additional Considerations: \n• Exercise caution, because even if you meet yield requirements in EMD, the risk could still be high for full-rate production quantities.","Lessons Learned: \n• Continually update yield and rate targets as more data becomes available and use the data to make informed program decisions and process improvements. \n• Not achieving targets by this time will demonstrate the need to closely monitor this effort and identify mitigations to minimize the impact of not achieving your targets."]
[322,8,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have the manufacturing processes to be used for Low Rate Initial Production (LRIP) been verified on a pilot line?","Purpose: \nTo use data from the pilot line environment to demonstrate that yields and rates will support production requirements. To implement a continuous improvement program using yield and rate data. The program needs verify the realism of achieving production requirements by using build data from a pilot line environment. To ensure we have confidence in our ability to achieve program requirements we should have achieved our production yield and rate targets at this time.","Additional Considerations: \n• Exercise caution, because even if you meet yield requirements in EMD, the risk could still be high for full-rate production quantities.\n• The contractor should implement a continuous improvement program using data generated from the production line.","Lessons Learned: \n• Continually update yield and rate targets as more data becomes available and use the data to make informed program decisions and process improvements."]
[323,8,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Does the data collected from pilot line builds meet the process capability (i.e., Cpk) targets? ","Purpose: \nTo use data from the pilot line environment to demonstrate that yields and rates will support production requirements. To implement a continuous improvement program using yield and rate data. The program needs verify the realism of achieving production requirements by using build data from a pilot line environment. To ensure we have confidence in our ability to achieve program requirements we should have achieved our production yield and rate targets at this time.","Additional Considerations: \n• Exercise caution, because even if you meet yield requirements in EMD, the risk could still be high for full-rate production quantities.\n• The contractor should implement a continuous improvement program using data generated from the production line.","Lessons Learned: \n• Continually update yield and rate targets as more data becomes available and use the data to make informed program decisions and process improvements."]
[324,8,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Based on the data collected from pilot line builds, have the requirements for process capabilities (i.e., Cpk) been refined for LRIP and FRP? ","Purpose: \nTo use data from the pilot line environment to demonstrate that yields and rates will support production requirements. To implement a continuous improvement program using yield and rate data. The program needs verify the realism of achieving production requirements by using build data from a pilot line environment. To ensure we have confidence in our ability to achieve program requirements we should have achieved our production yield and rate targets at this time.","Additional Considerations: \n• Exercise caution, because even if you meet yield requirements in EMD, the risk could still be high for full-rate production quantities.\n• The contractor should implement a continuous improvement program using data generated from the production line.","Lessons Learned: \n• Continually update yield and rate targets as more data becomes available and use the data to make informed program decisions and process improvements."]
[325,9,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Are manufacturing processes stable, capable, and adequately controlled (i.e., meeting Cpks) for Low Rate Initial Production (LRIP)?","Purpose: \nTo use demonstrated results from a LRIP line to verify you have achieved the target yield and rates to meet production rate requirements. To implement a continuous improvement program using yield and rate data. The program manager must verify the likelihood of achieving production requirements by using actual build data from LRIP. To ensure you have confidence in your ability to achieve program requirements you should have achieved either FRP production yield and rate targets or have a low risk plan in place to achieve them by this time.","Additional Considerations: \n• None","Lessons Learned: \n• Continually update yield and rate targets as more data becomes available and use the data to make informed program decisions and process improvements."]
[326,9,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have manufacturing processes achieved program Low Rate Initial Production (LRIP) objectives? ","Purpose: \nTo use demonstrated results from a LRIP line to verify you have achieved the target yield and rates to meet production rate requirements. To implement a continuous improvement program using yield and rate data. The program manager must verify the likelihood of achieving production requirements by using actual build data from LRIP. To ensure you have confidence in your ability to achieve program requirements you should have achieved either FRP production yield and rate targets or have a low risk plan in place to achieve them by this time.","Additional Considerations: \n• None","Lessons Learned: \n• Continually update yield and rate targets as more data becomes available and use the data to make informed program decisions and process improvements."]
[327,9,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have variability experiments been conducted to show impacts from Full Rate Production (FRP)?","Purpose: \nTo use demonstrated results from a LRIP line to verify you have achieved the target yield and rates to meet production rate requirements. To implement a continuous improvement program using yield and rate data. The program manager must verify the likelihood of achieving production requirements by using actual build data from LRIP. To ensure you have confidence in your ability to achieve program requirements you should have achieved either FRP production yield and rate targets or have a low risk plan in place to achieve them by this time.","Additional Considerations: \n• None","Lessons Learned: \n• Continually update yield and rate targets as more data becomes available and use the data to make informed program decisions and process improvements."]
[328,9,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have variability experiments been conducted to determine the potential for continuous improvement?  ","Purpose: \nTo verify FRP yield and rate targets have been achieved. The program manager must verify achieving production requirements by using the actual build data. To ensure confidence in our ability to achieve program requirements we should have achieved FRP production yield and rate targets or have a low risk plan in place to achieve them by this time.","Additional Considerations: \n• None","Lessons Learned: \n• Continually update yield and rate targets as more data becomes available and use the data to make continuous process improvements."]
[329,10,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Are manufacturing processes stable, capable, and adequately controlled (i.e., meeting Cpks)?","Purpose: \nTo verify FRP yield and rate targets have been achieved. The program manager must verify achieving production requirements by using the actual build data. To ensure confidence in our ability to achieve program requirements we should have achieved FRP production yield and rate targets or have a low risk plan in place to achieve them by this time.","Additional Considerations: \n• None","Lessons Learned: \n• Continually update yield and rate targets as more data becomes available and use the data to make continuous process improvements."]
[330,10,"E - Process Capability & Control","E.2 - Manufacturing Process Maturity","Have manufacturing processes achieved program rate production (FRP) objectives?","Purpose: \nTo lay the groundwork for ongoing Quality System development concurrent with system design, in parallel with the development of a manufacturing capability for the system. To document potential effects of quality on new products or manufacturing technologies and any technology development related to quality processes in the Technology Development Strategy. Assessment of manufacturing feasibility in the AoA is a requirement in the MSA phase for each of the Critical Technologies (CTs) in all possible material solutions. Manufacturing process yields and inspectability may be relevant considerations for some or all material solutions. \n","Additional Considerations: \n• Is there a plan to evaluate the quality strategy as part of Analysis of Alternatives (AoA)? \n• What data is available from similar, more mature, systems to assist in quality planning?\n• Are there any lessons learned on quality strategies from other similar system?\n• To properly assess risks and compare alternatives the effect of quality on cost, schedule and performance should not be ignored. \n","Lessons Learned: \n• Developing a Quality Strategy as early as possible in the lifecycle prevents potential misunderstandings about quality requirements. Identifying a need for a strategy addressing quality in specific phases, manufacturing or assembly conditions, or system components within an outlined quality strategy pre Milestone A will ensure the details are well considered and updated later in Engineering and Manufacturing Development (EMD).\n• Many program managers and chief engineers will have difficulty recognizing the need to address quality strategy this early in the program. Make sure they are brought up to speed on the logic for this step in the process, and keep them informed as the strategy is developed so they see the value added."]
[331,1,"E - Process Capability & Control","E.3 - Process Yields and Rates","Have hypotheses been developed regarding future state manufacturing yields and rates?","Purpose: \nTo lay the groundwork for ongoing Quality System development concurrent with system design, in parallel with the development of a manufacturing capability for the system. To document potential effects of quality on new products or manufacturing technologies and any technology development related to quality processes in the Technology Development Strategy. Assessment of manufacturing feasibility in the AoA is a requirement in the MSA phase for each of the Critical Technologies (CTs) in all possible material solutions. Manufacturing process yields and inspectability may be relevant considerations for some or all material solutions. \n","Additional Considerations: \n• Is there a plan to evaluate the quality strategy as part of Analysis of Alternatives (AoA)? \n• What data is available from similar, more mature, systems to assist in quality planning?\n• Are there any lessons learned on quality strategies from other similar system?\n• To properly assess risks and compare alternatives the effect of quality on cost, schedule and performance should not be ignored. \n","Lessons Learned: \n• Developing a Quality Strategy as early as possible in the lifecycle prevents potential misunderstandings about quality requirements. Identifying a need for a strategy addressing quality in specific phases, manufacturing or assembly conditions, or system components within an outlined quality strategy pre Milestone A will ensure the details are well considered and updated later in Engineering and Manufacturing Development (EMD).\n• Many program managers and chief engineers will have difficulty recognizing the need to address quality strategy this early in the program. Make sure they are brought up to speed on the logic for this step in the process, and keep them informed as the strategy is developed so they see the value added."]
[332,2,"E - Process Capability & Control","E.3 - Process Yields and Rates","Have studies been performed to test hypotheses regarding yields and rates?","Purpose: \nTo build on the groundwork already established for an ongoing Quality System development concurrent with the system design in parallel with the development of a manufacturing capability for the system. Consideration should be given to the compatibility of the existing Quality System with the quality requirements likely to apply to the product in question (e.g., ISO9001 or AS9100C). The potential effect of quality on new product or manufacturing technology and any technology development related to quality processes are documented in the Technology Development Strategy. The basic outline for the quality system is documented in the Systems Engineering Plan. Assessment of manufacturing feasibility in the AoA is a requirement for each of the Critical Technologies (CTs) in all possible material solutions. Further evaluation of technology demonstrations in the Technology Development phase provides greater opportunity to identify quality risks and opportunities to improve future process yields. Manufacturing process yields and inspectability may be relevant considerations for some or all material solutions.","Additional Considerations:\n• Is there a plan to evaluate the quality strategy as part of Analysis of Alternatives (AoA)? \n• What data is available from similar, more mature, systems to assist in quality planning?\n• Are there any lessons learned on quality strategies from other similar system?\n• To properly assess risks and compare alternatives, the effect of quality on cost, schedule and performance should not be ignored.","\n\nLessons Learned: \n• Developing a Quality Strategy as early as possible in the lifecycle prevents potential misunderstandings about quality requirements. Identifying a need for a strategy addressing quality in specific phases, manufacturing or assembly conditions, or system components within an outlined quality strategy pre-Milestone A will ensure the details are well considered and updated later in Engineering and Manufacturing Development (EMD).\n• Many program managers and chief engineers will have difficulty recognizing the need to address quality strategy this early in the program. Make sure they are brought up to speed on the logic for this step in the process, and keep them informed as the strategy is developed so they see the value added."]
[333,3,"E - Process Capability & Control","E.3 - Process Yields and Rates","Have initial estimates of yields and rates for system concepts been identified through laboratory experiments?","Purpose: \nTo establish criteria (thresholds) for evaluating Quality Management System(s) going into design and development reviews during EMD. To continue to develop an ongoing Quality System concurrent with the system design, and in parallel with the development of a manufacturing capability for the system. Since the Quality System utilized by the contractor is often based on AS 9100C or ISO9001, consideration should be given to compatibility of the existing Quality System with the quality requirements coming out of design development. Program Quality Plans are also created to document the integration of the program office’s quality processes with the contractor’s, depot’s, subcontractor’s and DCMA’s Quality systems. The identification of manufacturing risks and demonstration of processes in a production relevant environment to assure an affordable program is an exit requirement for the Technology Development phase. The Quality Management System is critical to evaluation of manufacturing performance, and costs associated with poor quality management can make a program unaffordable; therefore, preparing for the development of an effective Quality Management System is a critical element leading into EMD and milestone B. Inspectability may also be a consideration since failure to identify defects increases cost throughout the lifecycle of the system as are support costs driven by poor reliability caused by latent defects and uncontrolled manufacturing process variability.","Additional Considerations:\n• What data is available from similar, more mature, systems to assist in quality planning?\n• Does data from similar systems provide confidence in the QMS to be used on this program?\n• Are there any lessons learned on quality strategies from other similar system?\n• To properly assess risks in EMD, the effect of quality on cost, schedule and performance should not be ignored.","Lessons Learned: \n• Leading into milestone B, there will be greater attention to quality system development than during previous MRAs. Make optimum use of plans and information gleaned during earlier reviews, and make sure the plans for quality system improvement address all shortfalls and will result in a capable quality system by the end of EMD.\n• The ability to identify quality system shortfalls will depend on how much insight management gets from the review of quality metrics. Make sure the quality metrics package is complete, accurate, and timely to the needs of management. Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions."]
[334,3,"E - Process Capability & Control","E.3 - Process Yields and Rates","Have yield and rate gaps for system concepts been identified?"]
[335,4,"E - Process Capability & Control","E.3 - Process Yields and Rates","Have yield and rate assessments on the preferred materiel solution been completed and considered in the AoA? ","Purpose: \nTo establish criteria (thresholds) for evaluating Quality Management System(s) going into design and development reviews during EMD. To continue to develop an ongoing Quality System concurrent with the system design, and in parallel with the development of a manufacturing capability for the system. Since the Quality System utilized by the contractor is often based on AS 9100C or ISO9001, consideration should be given to compatibility of the existing Quality System with the quality requirements coming out of design development. Program Quality Plans are also created to document the integration of the program office’s quality processes with the contractor’s, depot’s, subcontractor’s and DCMA’s Quality systems. The identification of manufacturing risks and demonstration of processes in a production relevant environment to assure an affordable program is an exit requirement for the Technology Development phase. The Quality Management System is critical to evaluation of manufacturing performance, and costs associated with poor quality management can make a program unaffordable; therefore, preparing for the development of an effective Quality Management System is a critical element leading into EMD and milestone B. Inspectability may also be a consideration since failure to identify defects increases cost throughout the lifecycle of the system as are support costs driven by poor reliability caused by latent defects and uncontrolled manufacturing process variability.","Additional Considerations:\n• What data is available from similar, more mature, systems to assist in quality planning?\n• Does data from similar systems provide confidence in the QMS to be used on this program?\n• Are there any lessons learned on quality strategies from other similar system?\n• To properly assess risks in EMD, the effect of quality on cost, schedule and performance should not be ignored.","Lessons Learned: \n• Leading into milestone B, there will be greater attention to quality system development than during previous MRAs. Make optimum use of plans and information gleaned during earlier reviews, and make sure the plans for quality system improvement address all shortfalls and will result in a capable quality system by the end of EMD.\n• The ability to identify quality system shortfalls will depend on how much insight management gets from the review of quality metrics. Make sure the quality metrics package is complete, accurate, and timely to the needs of management. Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions."]
[336,4,"E - Process Capability & Control","E.3 - Process Yields and Rates","Have yield and rate gap closure strategies been identified for the preferred materiel solution and documented in the SEP?"]
[337,5,"E - Process Capability & Control","E.3 - Process Yields and Rates","Have target yield and rates been established for the pilot line, Low Rate Initial Production (LRIP), and Full Rate Production (FRP)?"]
[338,5,"E - Process Capability & Control","E.3 - Process Yields and Rates","Have any risks and issues arising from yields and rates been identified?","Purpose: \nTo establish criteria (thresholds) for evaluating Quality Management System(s) going into design and development reviews during EMD. To continue to develop an ongoing Quality System concurrent with the system design, and in parallel with the development of a manufacturing capability for the system. Since the Quality System utilized by the contractor is often based on AS 9100C or ISO9001, consideration should be given to compatibility of the existing Quality System with the quality requirements coming out of design development. Program Quality Plans are also created to document the integration of the program office’s quality processes with the contractor’s, depot’s, subcontractor’s and DCMA’s Quality systems. The identification of manufacturing risks and demonstration of processes in a production relevant environment to assure an affordable program is an exit requirement for the Technology Development phase. The Quality Management System is critical to evaluation of manufacturing performance, and costs associated with poor quality management can make a program unaffordable; therefore, preparing for the development of an effective Quality Management System is a critical element leading into EMD and milestone B. Inspectability may also be a consideration since failure to identify defects increases cost throughout the lifecycle of the system as are support costs driven by poor reliability caused by latent defects and uncontrolled manufacturing process variability.","Additional Considerations:\n• What data is available from similar, more mature, systems to assist in quality planning?\n• Does data from similar systems provide confidence in the QMS to be used on this program?\n• Are there any lessons learned on quality strategies from other similar system?\n• To properly assess risks in EMD, the effect of quality on cost, schedule and performance should not be ignored.","Lessons Learned: \n• Leading into milestone B, there will be greater attention to quality system development than during previous MRAs. Make optimum use of plans and information gleaned during earlier reviews, and make sure the plans for quality system improvement address all shortfalls and will result in a capable quality system by the end of EMD.\n• The ability to identify quality system shortfalls will depend on how much insight management gets from the review of quality metrics. Make sure the quality metrics package is complete, accurate, and timely to the needs of management. Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions."]
[339,5,"E - Process Capability & Control","E.3 - Process Yields and Rates","Have improvement plans for yields and rates been developed and initiated? ","Purpose: \nTo apply the quality targets/thresholds to pre-production data to evaluate Quality Management System(s) coming out of milestone B and going into design and development reviews during EMD. To continue to develop an ongoing Quality System concurrent with the system design, and in parallel with the development of a manufacturing capability for the system. Utilizing AS9100 C, to ensure the QMS meets industry standards applicable to the aerospace and defense system being developed. The basic outline for the quality system is documented in the Systems Engineering Plan, Program Quality Plans, Contractor(s) Quality Plans, and Memorandums of Understanding (MOUs) with DCMA and maintenance depots. Demonstration of a capable manufacturing system is required during EMD, and the QMS is an important element of the manufacturing system. The QMS is critical to evaluation of manufacturing performance, and the costs associated with poor quality management can make a program unaffordable.","Additional Considerations:\n• What quality targets are in place, and what is your status in achieving these targets?\n• Does data from similar systems provide confidence in the QMS which will be used on this program?\n• What progress has been made with regard to quality improvement initiatives?","Lessons Learned: \n• Coming out of milestone B, there should be some momentum regarding quality system development highlighted during the previous Manufacturing Readiness Assessment. Schedule regular reviews of progress to maturation plans and plans for quality system improvement to ensure the momentum isn’t lost after the milestone review. These quality improvements will help ensure a capable quality system by the end of EMD.\n• The ability to manage program quality will depend on how much insight management gets from the review of quality metrics. Make sure the quality metrics package is complete, accurate, and timely to the needs of management. Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions."]
[340,6,"E - Process Capability & Control","E.3 - Process Yields and Rates","Are yields and rates from the production relevant environment being evaluated against targets?","Purpose: \nTo apply the quality targets/thresholds to pre-production data to evaluate Quality Management System(s) coming out of milestone B and going into design and development reviews during EMD. To continue to develop an ongoing Quality System concurrent with the system design, and in parallel with the development of a manufacturing capability for the system. Utilizing AS9100 C, to ensure the QMS meets industry standards applicable to the aerospace and defense system being developed. The basic outline for the quality system is documented in the Systems Engineering Plan, Program Quality Plans, Contractor(s) Quality Plans, and Memorandums of Understanding (MOUs) with DCMA and maintenance depots. Demonstration of a capable manufacturing system is required during EMD, and the QMS is an important element of the manufacturing system. The QMS is critical to evaluation of manufacturing performance, and the costs associated with poor quality management can make a program unaffordable.","Additional Considerations:\n• What quality targets are in place, and what is your status in achieving these targets?\n• Does data from similar systems provide confidence in the QMS which will be used on this program?\n• What progress has been made with regard to quality improvement initiatives?","Lessons Learned: \n• Coming out of milestone B, there should be some momentum regarding quality system development highlighted during the previous Manufacturing Readiness Assessment. Schedule regular reviews of progress to maturation plans and plans for quality system improvement to ensure the momentum isn’t lost after the milestone review. These quality improvements will help ensure a capable quality system by the end of EMD.\n• The ability to manage program quality will depend on how much insight management gets from the review of quality metrics. Make sure the quality metrics package is complete, accurate, and timely to the needs of management. Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions."]
[341,6,"E - Process Capability & Control","E.3 - Process Yields and Rates","Are results of the evaluation of production relevant yields and rates included in the ongoing improvement plans?","Purpose: \nTo apply the quality targets/thresholds to pilot line data to evaluate the Quality Management System(s) coming out of milestone B and CDR. Continue to develop an ongoing Quality System concurrent with the system design, and in parallel with the development of a manufacturing capability for the system. Utilizing AS9100 C, to ensure the QMS meets industry standards applicable to the aerospace and defense system being developed. The basic outline for the quality system is documented in the Systems Engineering Plan, Program Quality Plans, Contractor(s) Quality Plans, and Memorandums of Understanding (MOUs) with DCMA and maintenance depots. Quality data should begin to make the case that maturity of the production system will soon justify entrance into LRIP, and then Full Rate Production. Demonstration of a capable manufacturing system is required during EMD, and the QMS is an important element of the manufacturing system. The QMS is critical to evaluation of manufacturing performance, and the costs associated with poor quality management can make a program unaffordable; therefore, the development of an effective Quality Management System is a critical element in EMD.","Additional Considerations:\n• What quality targets are in place, and what is your status in achieving these targets?\n• Does quality data from production representative and pilot line units provide confidence in the QMS which will be used on this program?\n• Is the QMS compliant with AS9100 C?","Lessons Learned: \n• Preparing for milestone C, there should be a great deal of program management attention to quality system development and resolution of issues identified during the previous MRA and the CDR. Continue to schedule regular reviews of progress to maturation plans and plans for quality system improvement to ensure the program won’t experience any surprises going into milestone C. These quality improvements will help ensure a capable quality system for LRIP and Full Rate Production.\n• The ability to manage program quality will depend on how much insight management gets from the review of quality metrics. Make sure the quality metrics package is complete, accurate, and timely to the needs of management. Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions."]
[342,7,"E - Process Capability & Control","E.3 - Process Yields and Rates","Are yields and rates from the production representative environment being evaluated against pilot line targets?","Purpose: \nTo apply the quality targets/thresholds to pilot line data to evaluate the Quality Management System(s) coming out of milestone B and CDR. Continue to develop an ongoing Quality System concurrent with the system design, and in parallel with the development of a manufacturing capability for the system. Utilizing AS9100 C, to ensure the QMS meets industry standards applicable to the aerospace and defense system being developed. The basic outline for the quality system is documented in the Systems Engineering Plan, Program Quality Plans, Contractor(s) Quality Plans, and Memorandums of Understanding (MOUs) with DCMA and maintenance depots. Quality data should begin to make the case that maturity of the production system will soon justify entrance into LRIP, and then Full Rate Production. Demonstration of a capable manufacturing system is required during EMD, and the QMS is an important element of the manufacturing system. The QMS is critical to evaluation of manufacturing performance, and the costs associated with poor quality management can make a program unaffordable; therefore, the development of an effective Quality Management System is a critical element in EMD.","Additional Considerations:\n• What quality targets are in place, and what is your status in achieving these targets?\n• Does quality data from production representative and pilot line units provide confidence in the QMS which will be used on this program?\n• Is the QMS compliant with AS9100 C?","Lessons Learned: \n• Preparing for milestone C, there should be a great deal of program management attention to quality system development and resolution of issues identified during the previous MRA and the CDR. Continue to schedule regular reviews of progress to maturation plans and plans for quality system improvement to ensure the program won’t experience any surprises going into milestone C. These quality improvements will help ensure a capable quality system for LRIP and Full Rate Production.\n• The ability to manage program quality will depend on how much insight management gets from the review of quality metrics. Make sure the quality metrics package is complete, accurate, and timely to the needs of management. Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions."]
[343,7,"E - Process Capability & Control","E.3 - Process Yields and Rates","Are results of an evaluation of the production representative yields and rates included in the ongoing improvement plans? "]
[344,8,"E - Process Capability & Control","E.3 - Process Yields and Rates","Have pilot line yields and rates been achieved?"]
[345,8,"E - Process Capability & Control","E.3 - Process Yields and Rates","Are the results of an evaluation of yields and rates from the pilot line being used to refine Low Rate Initial Production (LRIP) requirements?","Purpose: \nTo apply quality targets/thresholds to LRIP data to evaluate the Quality Management System(s) coming out of milestone C, to demonstrate the capability of the production and quality systems to handle full rate production. AS9100 C is the industry standard applicable to our QMS. The basic outline for the quality system is documented in the Systems Engineering Plan, Program Quality Plans, Contractor(s) Quality Plans, and Memorandums of Understanding (MOUs) with DCMA and maintenance depots. Quality data should indicate the level of maturity of the production system will soon justify entrance into Full Rate Production. Manufacturing processes must be under control at milestone C. During LRIP final manufacturing development will be completed, and the QMS is an important element of the manufacturing system. The QMS is also critical to evaluation of manufacturing performance, and the costs associated with a poor quality management system can make a program unaffordable. Therefore, the development of an effective Quality Management System is a critical element in Production.","Additional Considerations:\n• What quality targets are in place, and what is your status in achieving these targets?\n• Does quality data from LRIP units provide confidence in the QMS which will be used on this program?\n• Is the QMS compliant with AS9100 C?","Lessons Learned: \n• Preparing for Full Rate Production, there should be a great deal of program management attention to quality system development and resolution of issues identified during the previous MRA, CDR, and milestone reviews. Continue to schedule regular reviews of progress to maturation plans and plans for quality system improvement to ensure the program won’t experience any surprises going into Full Rate Production. These quality improvements will help ensure a capable quality system for LRIP and Full Rate Production.\n• The ability to manage program quality will depend on how much insight management gets from the review of quality metrics. Make sure the quality metrics package is complete, accurate, and timely to the needs of management. Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions."]
[346,8,"E - Process Capability & Control","E.3 - Process Yields and Rates","Are yield and rate improvement plans ongoing and up-to-date?","Purpose: \nTo apply quality targets/thresholds to LRIP data to evaluate the Quality Management System(s) coming out of milestone C, to demonstrate the capability of the production and quality systems to handle full rate production. AS9100 C is the industry standard applicable to our QMS. The basic outline for the quality system is documented in the Systems Engineering Plan, Program Quality Plans, Contractor(s) Quality Plans, and Memorandums of Understanding (MOUs) with DCMA and maintenance depots. Quality data should indicate the level of maturity of the production system will soon justify entrance into Full Rate Production. Manufacturing processes must be under control at milestone C. During LRIP final manufacturing development will be completed, and the QMS is an important element of the manufacturing system. The QMS is also critical to evaluation of manufacturing performance, and the costs associated with a poor quality management system can make a program unaffordable. Therefore, the development of an effective Quality Management System is a critical element in Production.","Additional Considerations:\n• What quality targets are in place, and what is your status in achieving these targets?\n• Does quality data from LRIP units provide confidence in the QMS which will be used on this program?\n• Is the QMS compliant with AS9100 C?","Lessons Learned: \n• Preparing for Full Rate Production, there should be a great deal of program management attention to quality system development and resolution of issues identified during the previous MRA, CDR, and milestone reviews. Continue to schedule regular reviews of progress to maturation plans and plans for quality system improvement to ensure the program won’t experience any surprises going into Full Rate Production. These quality improvements will help ensure a capable quality system for LRIP and Full Rate Production.\n• The ability to manage program quality will depend on how much insight management gets from the review of quality metrics. Make sure the quality metrics package is complete, accurate, and timely to the needs of management. Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions."]
[347,9,"E - Process Capability & Control","E.3 - Process Yields and Rates","Have the Low Rate Initial Production (LRIP) yield and rate targets been achieved?","Purpose: \nTo apply quality targets/thresholds to LRIP data to evaluate the Quality Management System(s) coming out of milestone C, to demonstrate the capability of the production and quality systems to handle full rate production. AS9100 C is the industry standard applicable to our QMS. The basic outline for the quality system is documented in the Systems Engineering Plan, Program Quality Plans, Contractor(s) Quality Plans, and Memorandums of Understanding (MOUs) with DCMA and maintenance depots. Quality data should indicate the level of maturity of the production system will soon justify entrance into Full Rate Production. Manufacturing processes must be under control at milestone C. During LRIP final manufacturing development will be completed, and the QMS is an important element of the manufacturing system. The QMS is also critical to evaluation of manufacturing performance, and the costs associated with a poor quality management system can make a program unaffordable. Therefore, the development of an effective Quality Management System is a critical element in Production.","Additional Considerations:\n• What quality targets are in place, and what is your status in achieving these targets?\n• Does quality data from LRIP units provide confidence in the QMS which will be used on this program?\n• Is the QMS compliant with AS9100 C?","Lessons Learned: \n• Preparing for Full Rate Production, there should be a great deal of program management attention to quality system development and resolution of issues identified during the previous MRA, CDR, and milestone reviews. Continue to schedule regular reviews of progress to maturation plans and plans for quality system improvement to ensure the program won’t experience any surprises going into Full Rate Production. These quality improvements will help ensure a capable quality system for LRIP and Full Rate Production.\n• The ability to manage program quality will depend on how much insight management gets from the review of quality metrics. Make sure the quality metrics package is complete, accurate, and timely to the needs of management. Hold metrics reviews frequently enough to get management accustomed to the use of quality metrics to make critical program decisions."]
[348,9,"E - Process Capability & Control","E.3 - Process Yields and Rates","Are the results of an evaluation of yields and rates from Low Rate Initial Production (LRIP) being used to refine rate production (FRP) requirements?","Purpose: \nTo continuously improve quality in support of, and throughout full rate production. LRIP data out of the QMS helps identify opportunities for continuous improvement. Quality data should indicate the level of maturity of the production system is commensurate with a program in Full Rate Production. For Full Rate Production, DODI 5000.02 requires demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes. The QMS is also critical to the collection of statistical data and demonstration of manufacturing process control. The philosophy of continuous improvement is essential to a successful production phase and a necessity for meeting cost targets and cost reduction. Data from the QMS drives continuous improvement during the production phase by focusing improvement efforts on high cost drivers and production processes with low yields. So an effective QMS is a critical element in Full Rate Production.","Additional Considerations:\n• What quality targets are in place, and what is your status in achieving these targets?\n• Does quality data from Rate Production units provide confidence in the QMS which will be used on this program?\n• Is there enough flexibility in the QMS to accommodate fluctuating production rates (accelerated production or program delay/stretched out production)?\n• Is the QMS compliant with AS9100 C?","Lessons Learned: \n• Following the Full Rate Production decision, there is likely to be a tendency to lose focus on continuous improvement. There may also be an experience drain as management is moved onto more high risk programs. Continue to schedule regular reviews of quality metrics and emphasize progress to quality system improvement initiatives to ensure the program’s quality doesn’t regress coming out of Full Rate Production. Educate any new management entering the program at this phase on the importance of continued focus on quality and continuous improvement.\n• The opportunities to reduce cost late in production are often accompanied with more quality risks than management recognizes. Make sure the impact on quality is addressed when considering production changes during this phase."]
[349,9,"E - Process Capability & Control","E.3 - Process Yields and Rates","Are yield and rate improvements on-going?","Purpose: \nTo continuously improve quality in support of, and throughout full rate production. LRIP data out of the QMS helps identify opportunities for continuous improvement. Quality data should indicate the level of maturity of the production system is commensurate with a program in Full Rate Production. For Full Rate Production, DODI 5000.02 requires demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes. The QMS is also critical to the collection of statistical data and demonstration of manufacturing process control. The philosophy of continuous improvement is essential to a successful production phase and a necessity for meeting cost targets and cost reduction. Data from the QMS drives continuous improvement during the production phase by focusing improvement efforts on high cost drivers and production processes with low yields. So an effective QMS is a critical element in Full Rate Production.","Additional Considerations:\n• What quality targets are in place, and what is your status in achieving these targets?\n• Does quality data from Rate Production units provide confidence in the QMS which will be used on this program?\n• Is there enough flexibility in the QMS to accommodate fluctuating production rates (accelerated production or program delay/stretched out production)?\n• Is the QMS compliant with AS9100 C?","Lessons Learned: \n• Following the Full Rate Production decision, there is likely to be a tendency to lose focus on continuous improvement. There may also be an experience drain as management is moved onto more high risk programs. Continue to schedule regular reviews of quality metrics and emphasize progress to quality system improvement initiatives to ensure the program’s quality doesn’t regress coming out of Full Rate Production. Educate any new management entering the program at this phase on the importance of continued focus on quality and continuous improvement.\n• The opportunities to reduce cost late in production are often accompanied with more quality risks than management recognizes. Make sure the impact on quality is addressed when considering production changes during this phase."]
[350,10,"E - Process Capability & Control","E.3 - Process Yields and Rates","Have the requirements for Full Rate Production (FRP) yields and rates been met?","Purpose: \nTo continuously improve quality in support of, and throughout full rate production. LRIP data out of the QMS helps identify opportunities for continuous improvement. Quality data should indicate the level of maturity of the production system is commensurate with a program in Full Rate Production. For Full Rate Production, DODI 5000.02 requires demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes. The QMS is also critical to the collection of statistical data and demonstration of manufacturing process control. The philosophy of continuous improvement is essential to a successful production phase and a necessity for meeting cost targets and cost reduction. Data from the QMS drives continuous improvement during the production phase by focusing improvement efforts on high cost drivers and production processes with low yields. So an effective QMS is a critical element in Full Rate Production.","Additional Considerations:\n• What quality targets are in place, and what is your status in achieving these targets?\n• Does quality data from Rate Production units provide confidence in the QMS which will be used on this program?\n• Is there enough flexibility in the QMS to accommodate fluctuating production rates (accelerated production or program delay/stretched out production)?\n• Is the QMS compliant with AS9100 C?","\n\n\nLessons Learned: \n• Following the Full Rate Production decision, there is likely to be a tendency to lose focus on continuous improvement. There may also be an experience drain as management is moved onto more high risk programs. Continue to schedule regular reviews of quality metrics and emphasize progress to quality system improvement initiatives to ensure the program’s quality doesn’t regress coming out of Full Rate Production. Educate any new management entering the program at this phase on the importance of continued focus on quality and continuous improvement.\n• The opportunities to reduce cost late in production are often accompanied with more quality risks than management recognizes. Make sure the impact on quality is addressed when considering production changes during this phase."]
[351,10,"E - Process Capability & Control","E.3 - Process Yields and Rates","Are yield improvements on-going?","Purpose: \nThe role of manufacturing in acquisition is to influence the design (make sure it is producible) and prepare for production. To assess manufacturing feasibility with a focus on product quality beginning with the quality strategy. The early phases of acquisition often provide opportunities for planning. At this time there may be no product, so it is a time for establishing the product inspection and acceptance testing strategy. The product quality strategy should be carried forward and become a part of the Technology Development Strategy (TDS) and Systems Engineering Plan (SEP).\n","Additional Considerations: \n• Does the strategy appropriately reflect the industry sector of the item developed and being produced?\n• Does the strategy appropriately reflect the rates/quantities of items estimated to be produced?\n• Is the strategy a “best practice?”\n","Lessons Learned: \n• Product quality must be appropriate for the item(s) being produced. For example, you should not expect to use statistical process control (SPC) on limited production items. Conversely, any product quality strategy that includes significant numbers of items should include a provision for SPC. \n• A tolerancing and dimensioning strategy should reflect the requirements of the end item. All too often engineers apply strategy of high tolerances just because the item is an aerospace item. Certainly many dimensional characteristics will need tight tolerances, but many characteristics perhaps could be looser and still provide for the requisite level of quality."]
[352,1,"F - Quality Management","F.1 - Quality Management","Have quality management considerations been surveyed and included in early planning activities?","Purpose: \nThe role of manufacturing in acquisition is to influence the design (make sure it is producible) and prepare for production. To assess manufacturing feasibility with a focus on product quality beginning with the quality strategy. The early phases of acquisition often provide opportunities for planning. At this time there may be no product, so it is a time for establishing the product inspection and acceptance testing strategy. The product quality strategy should be carried forward and become a part of the Technology Development Strategy (TDS) and Systems Engineering Plan (SEP).\n","Additional Considerations: \n• Does the strategy appropriately reflect the industry sector of the item developed and being produced?\n• Does the strategy appropriately reflect the rates/quantities of items estimated to be produced?\n• Is the strategy a “best practice?”\n","Lessons Learned: \n• Product quality must be appropriate for the item(s) being produced. For example, you should not expect to use statistical process control (SPC) on limited production items. Conversely, any product quality strategy that includes significant numbers of items should include a provision for SPC. \n• A tolerancing and dimensioning strategy should reflect the requirements of the end item. All too often engineers apply strategy of high tolerances just because the item is an aerospace item. Certainly many dimensional characteristics will need tight tolerances, but many characteristics perhaps could be looser and still provide for the requisite level of quality."]
[353,2,"F - Quality Management","F.1 - Quality Management","Have quality management needs been assessed, analyzed, and validated?","Purpose: \nTo influence the design (make sure it is producible) and prepare for production. To evaluate manufacturing processes. This includes the evaluation of product quality. Technology development results in the building of prototype components. It is logical to assume that the roles and responsibilities have been identified for acceptance test procedures to include in-process and final inspections. As the design matures one of the requirements is to identify key characteristics. Those are the product characteristics that are the most important and are the ones that might require the use of statistical process controls (SPC), if applicable. Product quality strategies and procedures should be carried forward and matured as product development moves from the component level to the system level.","Additional Considerations:\n• Were the prototype components built and inspected in a production relevant environment?\n• What constitutes a relevant environment for your product (system)?","\nLessons Learned: \n• A production relevant environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a production relevant environment for a satellite is different than that used in building prototype air vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control."]
[354,3,"F - Quality Management","F.1 - Quality Management","Have quality management requirements for system concepts been identified?","Purpose: \nTo influence the design (make sure it is producible) and prepare for production. To evaluate manufacturing processes. This includes the evaluation of product quality. Technology development results in the building of prototype components. It is logical to assume that the roles and responsibilities have been identified for acceptance test procedures to include in-process and final inspections. As the design matures one of the requirements is to identify key characteristics. Those are the product characteristics that are the most important and are the ones that might require the use of statistical process controls (SPC), if applicable. Product quality strategies and procedures should be carried forward and matured as product development moves from the component level to the system level.","Additional Considerations:\n• Were the prototype components built and inspected in a production relevant environment?\n• What constitutes a relevant environment for your product (system)?","\nLessons Learned: \n• A production relevant environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a production relevant environment for a satellite is different than that used in building prototype air vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control."]
[355,4,"F - Quality Management","F.1 - Quality Management","Has the quality strategy for the preferred materiel solution been developed?"]
[356,4,"F - Quality Management","F.1 - Quality Management","Has the quality strategy for the preferred materiel solution been considered in the AoA?","Purpose: \nTo influence the design (make sure it is producible) and prepare for production. To identify any remaining design, integration and manufacturing risks. The program is ready to exit the TD phase when: (1) The technology and manufacturing processes for that program have been assessed and demonstrated in a relevant environment, and (2) all manufacturing risks have been identified. Technology development results in the building of systems and the integration of subsystems and components. Roles and responsibilities should have been identified for acceptance test procedures to include in-process and final inspections at the system level. Key characteristics should have been identified. Those product characteristics are the most important and might require the use of statistical process controls (SPC). Product quality strategies and procedures developed in TD should be carried forward and matured as product development moves into EMD.","Additional Considerations:\n• Have product quality risks been identified?\n• Is the risk management plan for those risks acceptable (within the scope of the current contract)?\n• Were the prototype systems built and inspected in a production relevant environment?\n• What constitutes a relevant environment for your product (system)?\n","Lessons Learned: \n• A production relevant environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a production relevant environment for a satellite is different than that used in building prototype air vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[357,4,"F - Quality Management","F.1 - Quality Management","Has the quality strategy for the preferred materiel solution been documented in the SEP and the AS?","Purpose: \nTo influence the design (make sure it is producible) and prepare for production. To identify any remaining design, integration and manufacturing risks. The program is ready to exit the TD phase when: (1) The technology and manufacturing processes for that program have been assessed and demonstrated in a relevant environment, and (2) all manufacturing risks have been identified. Technology development results in the building of systems and the integration of subsystems and components. Roles and responsibilities should have been identified for acceptance test procedures to include in-process and final inspections at the system level. Key characteristics should have been identified. Those product characteristics are the most important and might require the use of statistical process controls (SPC). Product quality strategies and procedures developed in TD should be carried forward and matured as product development moves into EMD.","Additional Considerations:\n• Have product quality risks been identified?\n• Is the risk management plan for those risks acceptable (within the scope of the current contract)?\n• Were the prototype systems built and inspected in a production relevant environment?\n• What constitutes a relevant environment for your product (system)?\n","Lessons Learned: \n• A production relevant environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a production relevant environment for a satellite is different than that used in building prototype air vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[358,5,"F - Quality Management","F.1 - Quality Management","Does the quality strategy include activities for Key Characteristic (KC) identification?"]
[359,6,"F - Quality Management","F.1 - Quality Management","Is an initial Quality Plan (QP) in place?","Purpose: \nTo influence the design (make sure it is producible) and prepare for production. To identify any remaining design, integration and manufacturing risks. The program is ready to exit the TD phase when: (1) The technology and manufacturing processes for that program have been assessed and demonstrated in a relevant environment, and (2) all manufacturing risks have been identified. Technology development results in the building of systems and the integration of subsystems and components. Roles and responsibilities should have been identified for acceptance test procedures to include in-process and final inspections at the system level. Key characteristics should have been identified. Those product characteristics are the most important and might require the use of statistical process controls (SPC). Product quality strategies and procedures developed in TD should be carried forward and matured as product development moves into EMD.","Additional Considerations:\n• Have product quality risks been identified?\n• Is the risk management plan for those risks acceptable (within the scope of the current contract)?\n• Were the prototype systems built and inspected in a production relevant environment?\n• What constitutes a relevant environment for your product (system)?\n","Lessons Learned: \n• A production relevant environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a production relevant environment for a satellite is different than that used in building prototype air vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[360,6,"F - Quality Management","F.1 - Quality Management","Is a Quality Management System (QMS) in place?","Purpose: \nTo identify and mitigate technical risks. To collect quality data in a production representative environment and use the results to shape improvement plans. Since CDR has been completed, 90% of the engineering drawings should have been released and the design or product baseline is now essentially frozen. The contractor’s ability to influence the design is extremely limited at this point as are opportunities to improve performance or reduce costs. A successful CDR will assess and assure that the critical manufacturing processes (including quality) are mature. Engineering and Manufacturing Development (EMD) results in the completion of the design, or at least, the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality.","Additional Considerations:\n• Have product quality targets been established?\n• Do product quality program requirements meet industry standards (control of non-conforming material, corrective action, product handling and storage, inspection records, etc.)?\nAdditional Considerations:\n• Have product quality targets been established?\n• Do product quality program requirements meet industry standards (control of non-conforming material, corrective action, product handling and storage, inspection records, etc.)?\n","Lessons Learned: \n• A production representative environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a production relevant environment for a satellite is different than that used in building prototype air vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Manufacturing and QA personnel should be involved in the CDR and in any follow-up activities."]
[361,6,"F - Quality Management","F.1 - Quality Management","Have risks and issues impacting quality been identified?","Purpose: \nTo identify and mitigate technical risks. To collect quality data in a production representative environment and use the results to shape improvement plans. Since CDR has been completed, 90% of the engineering drawings should have been released and the design or product baseline is now essentially frozen. The contractor’s ability to influence the design is extremely limited at this point as are opportunities to improve performance or reduce costs. A successful CDR will assess and assure that the critical manufacturing processes (including quality) are mature. Engineering and Manufacturing Development (EMD) results in the completion of the design, or at least, the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality.","Additional Considerations:\n• Have product quality targets been established?\n• Do product quality program requirements meet industry standards (control of non-conforming material, corrective action, product handling and storage, inspection records, etc.)?\nAdditional Considerations:\n• Have product quality targets been established?\n• Do product quality program requirements meet industry standards (control of non-conforming material, corrective action, product handling and storage, inspection records, etc.)?\n","Lessons Learned: \n• A production representative environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a production relevant environment for a satellite is different than that used in building prototype air vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Manufacturing and QA personnel should be involved in the CDR and in any follow-up activities."]
[362,6,"F - Quality Management","F.1 - Quality Management","Have quality metrics been identified?","Purpose: \nTo identify and mitigate technical risks. To collect quality data in a production representative environment and use the results to shape improvement plans. Since CDR has been completed, 90% of the engineering drawings should have been released and the design or product baseline is now essentially frozen. The contractor’s ability to influence the design is extremely limited at this point as are opportunities to improve performance or reduce costs. A successful CDR will assess and assure that the critical manufacturing processes (including quality) are mature. Engineering and Manufacturing Development (EMD) results in the completion of the design, or at least, the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality.","Additional Considerations:\n• Have product quality targets been established?\n• Do product quality program requirements meet industry standards (control of non-conforming material, corrective action, product handling and storage, inspection records, etc.)?\nAdditional Considerations:\n• Have product quality targets been established?\n• Do product quality program requirements meet industry standards (control of non-conforming material, corrective action, product handling and storage, inspection records, etc.)?\n","Lessons Learned: \n• A production representative environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a production relevant environment for a satellite is different than that used in building prototype air vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• Many contractors do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Manufacturing and QA personnel should be involved in the CDR and in any follow-up activities."]
[363,6,"F - Quality Management","F.1 - Quality Management","Have improvement plans to mitigate quality risks and issues been initiated?"]
[364,7,"F - Quality Management","F.1 - Quality Management","Have the quality targets been established?","Purpose: \nTo reduce system-level risk. To integrate components and subsystems, and complete the detailed design to meet performance requirements with a producible and sustainable design. Engineering and Manufacturing Development (EMD) typically includes the demonstration of production prototype articles or engineering development models. EMD results in the completion of the design, at least the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality.","Additional Considerations:\n• Are measurement procedures and controls in place (e.g., SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Non-conforming material is properly identified and corrective actions taken.\n• Inspection status is clearly identified.\n• Product quality records are available and meet appropriate requirements.\n• Have quality risks and adequate metrics been identified?\n• Have quality targets been demonstrated on the pilot line?\n• Is continuous quality improvement on-going?","Lessons Learned: \n• A pilot line environment is different for different products which is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a pilot line environment for a satellite is different than that used in building prototype air vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Some programs are so low volume that their “Full Rate Production” may be only one or two items (e.g., space programs) in which case, the pilot line becomes, in essence, the FRP line. Also, because of low-rate production, there may not be sufficient data to do a formal yield plot for key characteristics.\n• Even though some programs are low volume (e.g., satellites or certain aircraft) there may be individual manufacturing processes used in production that are repeated sufficiently to gather yield date (e.g., manufacture and installation of solar cells, riveting of fuselage skins, soldering of wiring harnesses to connectors, etc.)\n• FRACAS reports can be a valuable source of information indicating potential process issues (inconsistent or variable processes impact product reliability).\n• Look for Pareto charts of major defects uncovered during inspection to identify process problems."]
[365,7,"F - Quality Management","F.1 - Quality Management","Do elements of the Quality Management System (QMS), such as control of nonconforming material, corrective action, etc., meet requirements of appropriate industry standards?","Purpose: \nTo reduce system-level risk. To integrate components and subsystems, and complete the detailed design to meet performance requirements with a producible and sustainable design. Engineering and Manufacturing Development (EMD) typically includes the demonstration of production prototype articles or engineering development models. EMD results in the completion of the design, at least the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality.","Additional Considerations:\n• Are measurement procedures and controls in place (e.g., SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Non-conforming material is properly identified and corrective actions taken.\n• Inspection status is clearly identified.\n• Product quality records are available and meet appropriate requirements.\n• Have quality risks and adequate metrics been identified?\n• Have quality targets been demonstrated on the pilot line?\n• Is continuous quality improvement on-going?","Lessons Learned: \n• A pilot line environment is different for different products which is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a pilot line environment for a satellite is different than that used in building prototype air vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Some programs are so low volume that their “Full Rate Production” may be only one or two items (e.g., space programs) in which case, the pilot line becomes, in essence, the FRP line. Also, because of low-rate production, there may not be sufficient data to do a formal yield plot for key characteristics.\n• Even though some programs are low volume (e.g., satellites or certain aircraft) there may be individual manufacturing processes used in production that are repeated sufficiently to gather yield date (e.g., manufacture and installation of solar cells, riveting of fuselage skins, soldering of wiring harnesses to connectors, etc.)\n• FRACAS reports can be a valuable source of information indicating potential process issues (inconsistent or variable processes impact product reliability).\n• Look for Pareto charts of major defects uncovered during inspection to identify process problems."]
[366,7,"F - Quality Management","F.1 - Quality Management","Has a program-specific Quality Program Plan been developed?","Purpose: \nTo reduce system-level risk. To integrate components and subsystems, and complete the detailed design to meet performance requirements with a producible and sustainable design. Engineering and Manufacturing Development (EMD) typically includes the demonstration of production prototype articles or engineering development models. EMD results in the completion of the design, at least the near completion of the design. Major design issues are well within program constraints and should have limited impact on product quality.","Additional Considerations:\n• Are measurement procedures and controls in place (e.g., SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Non-conforming material is properly identified and corrective actions taken.\n• Inspection status is clearly identified.\n• Product quality records are available and meet appropriate requirements.\n• Have quality risks and adequate metrics been identified?\n• Have quality targets been demonstrated on the pilot line?\n• Is continuous quality improvement on-going?","Lessons Learned: \n• A pilot line environment is different for different products which is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a pilot line environment for a satellite is different than that used in building prototype air vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Some programs are so low volume that their “Full Rate Production” may be only one or two items (e.g., space programs) in which case, the pilot line becomes, in essence, the FRP line. Also, because of low-rate production, there may not be sufficient data to do a formal yield plot for key characteristics.\n• Even though some programs are low volume (e.g., satellites or certain aircraft) there may be individual manufacturing processes used in production that are repeated sufficiently to gather yield date (e.g., manufacture and installation of solar cells, riveting of fuselage skins, soldering of wiring harnesses to connectors, etc.)\n• FRACAS reports can be a valuable source of information indicating potential process issues (inconsistent or variable processes impact product reliability).\n• Look for Pareto charts of major defects uncovered during inspection to identify process problems."]
[367,8,"F - Quality Management","F.1 - Quality Management","Has the program-specific Quality Program Plan been established?"]
[368,8,"F - Quality Management","F.1 - Quality Management","Has a program-specific Quality Manager been assigned?"]
[369,8,"F - Quality Management","F.1 - Quality Management","Have the previously established quality targets for pilot line been met? ","Purpose: \nTo provide an adequate and efficient manufacturing capability. To achieve operational capability to satisfy mission needs. This effort should result in completion of manufacturing development to ensure capability to produce the minimum quantity of production-representative articles for IOT&E, establish an initial production base for the system, and permit orderly increase in production rate leading to full-rate production when IOT&E is successfully completed.","Additional Considerations:\n• Are measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Non-conforming material is properly identified and corrective actions taken.\n• Inspection status is clearly identified.\n• Product quality records are available and meet appropriate requirements.","Lessons Learned: \n• A low-rate initial production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. For example, a pilot line environment for a satellite is different than that used in building prototype air vehicles.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[370,8,"F - Quality Management","F.1 - Quality Management","Have results of an evaluation of pilot line quality data actuals against quality targets been used for continuous quality improvement? ","Purpose: \nTo provide operational items to the warfighter and achieve an operational capability that satisfies mission needs. The knowledge required to support system approval must include demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes.","Additional Considerations:\n• Are measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Non-conforming material is properly identified and corrective actions taken.\n• Inspection status is clearly identified.\n• Product quality records are available and meet appropriate requirements.\n• Is the contractor conducting continuous improvement activities?","Lessons Learned: \n• A full-rate production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[371,9,"F - Quality Management","F.1 - Quality Management","Have quality targets been verified on the Low Rate Initial Production (LRIP) line?","Purpose: \nTo provide operational items to the warfighter and achieve an operational capability that satisfies mission needs. The knowledge required to support system approval must include demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes.","Additional Considerations:\n• Are measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Non-conforming material is properly identified and corrective actions taken.\n• Inspection status is clearly identified.\n• Product quality records are available and meet appropriate requirements.\n• Is the contractor conducting continuous improvement activities?","Lessons Learned: \n• A full-rate production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[372,9,"F - Quality Management","F.1 - Quality Management","Is continuous quality improvement on-going?","Purpose: \nTo provide operational items to the warfighter and achieve an operational capability that satisfies mission needs. The knowledge required to support system approval must include demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes.","Additional Considerations:\n• Are measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Non-conforming material is properly identified and corrective actions taken.\n• Inspection status is clearly identified.\n• Product quality records are available and meet appropriate requirements.\n• Is the contractor conducting continuous improvement activities?","Lessons Learned: \n• A full-rate production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[373,9,"F - Quality Management","F.1 - Quality Management","Is management review of quality measures conducted on a regular basis with appropriate actions being taken?","Purpose: \nTo begin assessing the quality capability of potential suppliers and to understand the supplier quality issues/risk in the AoAs. To use this information to understand the risk in selecting the alternatives to proceed with into TD. To use this information to quantify the uncertainty/risk of the achieving the program objectives. The program manager must understand the capability of the supplier base to provide quality products for each alternative design being considered and reflect risk in program planning efforts. It is important to survey potential suppliers’ quality systems capability to meet program requirements and identify risk presented by each supplier. At the end of this phase you will select the best alternatives available to address the user’s requirements to proceed into TD.","Additional Considerations: \n• Do the suppliers’ quality systems appropriately reflect the industry sector of the item developed and being produced?\n• Does the suppliers’ quality strategy appropriately reflect the rates/quantities of items estimated to be produced?\n• Is the supplier quality strategy a “best practice?”\n","Lessons Learned: \n• Supplier product quality must be appropriate for the item(s) being produced. For example, you should not expect to use statistical process control (SPC) on limited production items. Conversely, any product quality strategy that includes significant numbers of items should include a provision for SPC. \n• A supplier tolerancing and dimensioning strategy should reflect the requirements of the end item. All too often engineers apply strategy of high tolerances just because the item is an aerospace item. Certainly many dimensional characteristics will need tight tolerances, but many characteristics perhaps could be looser and still provide for the requisite level of quality."]
[374,10,"F - Quality Management","F.1 - Quality Management","Have the previously established quality targets for Full Rate Production (FRP) been verified by actual quality data?","Purpose: \nTo assess the quality capability of suppliers producing prototype items and identify supplier quality issues/risk. To use this information to quantify the uncertainty/risk of the achieving the program objectives and implement mitigation plans to reduce that risk. The program manager must assess supplier quality capabilities against program requirements and institute plans to achieve the quality requirements.","Additional Considerations: \n• Were the supplier prototype items built and inspected in a production relevant environment?\n• What constitutes a relevant environment for your supplier","\nLessons Learned: \n• A production relevant environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. Supplier quality systems must be validated to ensure they are using production relevant processes.\n• Many suppliers do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control. \n• Programs have been shut down because of supplier QA issues."]
[375,10,"F - Quality Management","F.1 - Quality Management","Is continuous quality improvement on-going?"]
[376,10,"F - Quality Management","F.1 - Quality Management","Are statistical controls being applied where appropriate?","Purpose: \nTo address supplier quality system issues/concerns building prototype items in a production relevant environment. To use this data to make improvement plans to achieve future requirements. The program needs to understand suppliers’ quality systems and capabilities vs. requirements. All issues/concerns need to have mitigation plans in place. If this is not addressed before entering EMD, it could become a major problem downstream.","Additional Considerations: \n• Have supplier product quality risks been identified?\n• Is the risk management plan for supplier risks acceptable (within the scope of the current contract)?\n• Were supplier prototype systems built and inspected in a production relevant environment?\n• What constitutes a relevant environment for your supplier?\n","Lessons Learned: \n• A production relevant environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. Supplier quality systems must be validated to ensure they are using production relevant processes.\n• Statistical Process Control is not a requirement for all suppliers. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• Many suppliers do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[377,1,"F - Quality Management","F.2 - Product Quality","Has the quality metrology state of the art been surveyed?"]
[378,1,"F - Quality Management","F.2 - Product Quality","Have hypotheses been developed regarding cause-effect relationships between technology variables and quality?","Purpose: \nTo focus on key suppliers and ensure they are using appropriate QMS. To use the data generated from building units in a production representative environment to compare capability versus program requirements. To ensure prime has a strategy in place to assess critical suppliers’ in-place. The program manager must consistently monitor the performance and capability of key/major suppliers.","Additional Considerations: \n• Have supplier product quality targets been established?\n• Do supplier quality program requirements meet industry standards (control of non-conforming material, corrective action, product handling and storage, inspection records, etc.)?","Lessons Learned: \n• A production representative environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. Supplier quality systems must be validated to ensure they are using production representative processes.\n• Statistical Process Control is not a requirement for all suppliers. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• Many suppliers do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow-down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• We need to continually assess the systems and capabilities of our suppliers as we progress into production. Historically a large percent of the cost and problems have come from suppliers and being informed in this area is critical in achieving program objectives. At this point the Prime contractor should be aggressively engaging with their major and critical subcontractors and vendors where known or potential risks may reside. Subcontractor QA is now a contact sport.\n• The prime contractor needs a system in place to ensure that appropriate quality requirements get flowed down to suppliers through the entire Work Breakdown Structure (WBS)."]
[379,2,"F - Quality Management","F.2 - Product Quality","Have studies been performed to test hypotheses regarding cause-effect relationships between technology variables and quality? ","Purpose: \nTo focus on key suppliers and ensure they are using appropriate QMS. To use the data generated from building units in a production representative environment to compare capability versus program requirements. To ensure prime has a strategy in place to assess critical suppliers’ in-place. The program manager must consistently monitor the performance and capability of key/major suppliers.","Additional Considerations: \n• Have supplier product quality targets been established?\n• Do supplier quality program requirements meet industry standards (control of non-conforming material, corrective action, product handling and storage, inspection records, etc.)?","Lessons Learned: \n• A production representative environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. Supplier quality systems must be validated to ensure they are using production representative processes.\n• Statistical Process Control is not a requirement for all suppliers. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• Many suppliers do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow-down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• We need to continually assess the systems and capabilities of our suppliers as we progress into production. Historically a large percent of the cost and problems have come from suppliers and being informed in this area is critical in achieving program objectives. At this point the Prime contractor should be aggressively engaging with their major and critical subcontractors and vendors where known or potential risks may reside. Subcontractor QA is now a contact sport.\n• The prime contractor needs a system in place to ensure that appropriate quality requirements get flowed down to suppliers through the entire Work Breakdown Structure (WBS)."]
[380,2,"F - Quality Management","F.2 - Product Quality","Have elements been identified which have a potential impact on quality (i.e., materials, processes, capabilities, limitations)?","Purpose: \nTo focus on key suppliers and ensure they are using appropriate QMS. To use the data generated from building units in a production representative environment to compare capability versus program requirements. To ensure prime has a strategy in place to assess critical suppliers’ in-place. The program manager must consistently monitor the performance and capability of key/major suppliers.","Additional Considerations: \n• Have supplier product quality targets been established?\n• Do supplier quality program requirements meet industry standards (control of non-conforming material, corrective action, product handling and storage, inspection records, etc.)?","Lessons Learned: \n• A production representative environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. Supplier quality systems must be validated to ensure they are using production representative processes.\n• Statistical Process Control is not a requirement for all suppliers. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• Many suppliers do not understand the concept of “key characteristics” and often do not identify them during the design process, and thus they do not know what feature or quality characteristic is most important to control. \n• There should be a clear path of requirements flow-down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• We need to continually assess the systems and capabilities of our suppliers as we progress into production. Historically a large percent of the cost and problems have come from suppliers and being informed in this area is critical in achieving program objectives. At this point the Prime contractor should be aggressively engaging with their major and critical subcontractors and vendors where known or potential risks may reside. Subcontractor QA is now a contact sport.\n• The prime contractor needs a system in place to ensure that appropriate quality requirements get flowed down to suppliers through the entire Work Breakdown Structure (WBS)."]
[381,3,"F - Quality Management","F.2 - Product Quality","Have system concept elements been evaluated for quality using experiments, modeling and simulation?","Purpose:\nTo reduce system-level risk. Through the conduct of systems engineering, the efforts in this phase integrate components and subsystems and complete the detailed design to meet performance requirements with a producible and sustainable design. EMD typically includes the demonstration of production prototype articles or engineering development models. Major design issues are well within program constraints and have limited impact on product quality.","\nAdditional Considerations:\n• What supplier quality targets are in place, and what is your status in achieving these targets?\n• Does quality data from supplier production representative and pilot line units provide confidence in the QMS which will be used on this program?\n• Are measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Is supplier non-conforming material properly identified and corrective actions taken?\n• Are supplier product quality records are available and meet appropriate requirements?\n• Have quality risks and adequate metrics been identified?\n• Have quality targets been demonstrated on the pilot line?\n• Is continuous quality improvement on-going at the supplier?\n","Lessons Learned: \n• A pilot line environment is different for different products which is often dependent on rates and quantities of items being produced and the complexity of those items. Supplier quality systems must be validated to ensure they are using production processes.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Even though some programs are low volume (e.g., satellites or certain aircraft) there may be individual manufacturing processes used in production that are repeated sufficiently to gather yield date (e.g., manufacture and installation of solar cells, riveting of fuselage skins, soldering of wiring harnesses to connectors, etc.)\n• Supplier FRACAS reports can be a valuable source of information indicating potential process issues (inconsistent or variable processes impact product reliability).\n• Look for supplier Pareto charts of major defects uncovered during inspection to identify process problems."]
[382,3,"F - Quality Management","F.2 - Product Quality","Have initial product quality requirements, risks, and issues been identified?","Purpose:\nTo reduce system-level risk. Through the conduct of systems engineering, the efforts in this phase integrate components and subsystems and complete the detailed design to meet performance requirements with a producible and sustainable design. EMD typically includes the demonstration of production prototype articles or engineering development models. Major design issues are well within program constraints and have limited impact on product quality.","\nAdditional Considerations:\n• What supplier quality targets are in place, and what is your status in achieving these targets?\n• Does quality data from supplier production representative and pilot line units provide confidence in the QMS which will be used on this program?\n• Are measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Is supplier non-conforming material properly identified and corrective actions taken?\n• Are supplier product quality records are available and meet appropriate requirements?\n• Have quality risks and adequate metrics been identified?\n• Have quality targets been demonstrated on the pilot line?\n• Is continuous quality improvement on-going at the supplier?\n","\nLessons Learned: \n• A pilot line environment is different for different products which is often dependent on rates and quantities of items being produced and the complexity of those items. Supplier quality systems must be validated to ensure they are using production processes.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Even though some programs are low volume (e.g., satellites or certain aircraft) there may be individual manufacturing processes used in production that are repeated sufficiently to gather yield date (e.g., manufacture and installation of solar cells, riveting of fuselage skins, soldering of wiring harnesses to connectors, etc.)\n• Supplier FRACAS reports can be a valuable source of information indicating potential process issues (inconsistent or variable processes impact product reliability).\n• Look for supplier Pareto charts of major defects uncovered during inspection to identify process problems."]
[383,3,"F - Quality Management","F.2 - Product Quality","Have inspection technologies been identified?","Purpose:\nTo reduce system-level risk. Through the conduct of systems engineering, the efforts in this phase integrate components and subsystems and complete the detailed design to meet performance requirements with a producible and sustainable design. EMD typically includes the demonstration of production prototype articles or engineering development models. Major design issues are well within program constraints and have limited impact on product quality.","\nAdditional Considerations:\n• What supplier quality targets are in place, and what is your status in achieving these targets?\n• Does quality data from supplier production representative and pilot line units provide confidence in the QMS which will be used on this program?\n• Are measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Is supplier non-conforming material properly identified and corrective actions taken?\n• Are supplier product quality records are available and meet appropriate requirements?\n• Have quality risks and adequate metrics been identified?\n• Have quality targets been demonstrated on the pilot line?\n• Is continuous quality improvement on-going at the supplier?\n","Lessons Learned: \n• A pilot line environment is different for different products which is often dependent on rates and quantities of items being produced and the complexity of those items. Supplier quality systems must be validated to ensure they are using production processes.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Even though some programs are low volume (e.g., satellites or certain aircraft) there may be individual manufacturing processes used in production that are repeated sufficiently to gather yield date (e.g., manufacture and installation of solar cells, riveting of fuselage skins, soldering of wiring harnesses to connectors, etc.)\n• Supplier FRACAS reports can be a valuable source of information indicating potential process issues (inconsistent or variable processes impact product reliability).\n• Look for supplier Pareto charts of major defects uncovered during inspection to identify process problems."]
[384,4,"F - Quality Management","F.2 - Product Quality","Have product quality requirements and the inspection and acceptance testing strategy for the preferred materiel solution been considered in AoA?","Purpose:\nTo reduce system-level risk. Through the conduct of systems engineering, the efforts in this phase integrate components and subsystems and complete the detailed design to meet performance requirements with a producible and sustainable design. EMD typically includes the demonstration of production prototype articles or engineering development models. Major design issues are well within program constraints and have limited impact on product quality.","\nAdditional Considerations:\n• What supplier quality targets are in place, and what is your status in achieving these targets?\n• Does quality data from supplier production representative and pilot line units provide confidence in the QMS which will be used on this program?\n• Are measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Is supplier non-conforming material properly identified and corrective actions taken?\n• Are supplier product quality records are available and meet appropriate requirements?\n• Have quality risks and adequate metrics been identified?\n• Have quality targets been demonstrated on the pilot line?\n• Is continuous quality improvement on-going at the supplier?\n","\nLessons Learned: \n• A pilot line environment is different for different products which is often dependent on rates and quantities of items being produced and the complexity of those items. Supplier quality systems must be validated to ensure they are using production processes.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Even though some programs are low volume (e.g., satellites or certain aircraft) there may be individual manufacturing processes used in production that are repeated sufficiently to gather yield date (e.g., manufacture and installation of solar cells, riveting of fuselage skins, soldering of wiring harnesses to connectors, etc.)\n• Supplier FRACAS reports can be a valuable source of information indicating potential process issues (inconsistent or variable processes impact product reliability).\n• Look for supplier Pareto charts of major defects uncovered during inspection to identify process problems."]
[385,4,"F - Quality Management","F.2 - Product Quality","Have product quality requirements and the inspection and acceptance testing strategy for the preferred materiel solution been documented in the AS?","Purpose:\nTo reduce system-level risk. Through the conduct of systems engineering, the efforts in this phase integrate components and subsystems and complete the detailed design to meet performance requirements with a producible and sustainable design. EMD typically includes the demonstration of production prototype articles or engineering development models. Major design issues are well within program constraints and have limited impact on product quality.","\nAdditional Considerations:\n• What supplier quality targets are in place, and what is your status in achieving these targets?\n• Does quality data from supplier production representative and pilot line units provide confidence in the QMS which will be used on this program?\n• Are measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Is supplier non-conforming material properly identified and corrective actions taken?\n• Are supplier product quality records are available and meet appropriate requirements?\n• Have quality risks and adequate metrics been identified?\n• Have quality targets been demonstrated on the pilot line?\n• Is continuous quality improvement on-going at the supplier?\n","\nLessons Learned: \n• A pilot line environment is different for different products which is often dependent on rates and quantities of items being produced and the complexity of those items. Supplier quality systems must be validated to ensure they are using production processes.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow-down from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate.\n• Even though some programs are low volume (e.g., satellites or certain aircraft) there may be individual manufacturing processes used in production that are repeated sufficiently to gather yield date (e.g., manufacture and installation of solar cells, riveting of fuselage skins, soldering of wiring harnesses to connectors, etc.)\n• Supplier FRACAS reports can be a valuable source of information indicating potential process issues (inconsistent or variable processes impact product reliability).\n• Look for supplier Pareto charts of major defects uncovered during inspection to identify process problems."]
[386,4,"F - Quality Management","F.2 - Product Quality","Have product quality risk and issue mitigation plans been documented in the SEP?"]
[387,5,"F - Quality Management","F.2 - Product Quality","Have roles and responsibilities for acceptance test procedures been identified?","Purpose: \nTo use demonstrated results from a LRIP line to verify adequate supplier controls and capability exists to meet FRP requirements. To ensure that continued audits and reviews are performed on suppliers as appropriate. The program manager must verify capability and controls from suppliers by using actual build data from LRIP. This early production data will lay the foundation for future production efforts. Management emphasis must be placed at suppliers where necessary to ensure production objectives can be achieved.","Additional Considerations: \n• Are supplier measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Is supplier non-conforming material is properly identified and corrective actions taken?\n• Supplier inspection status is clearly identified.\n• Supplier product quality records are available and meet appropriate requirements.","Lessons Learned: \n• A low-rate initial production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. Supplier quality systems must be validated to ensure they are using production processes.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[388,5,"F - Quality Management","F.2 - Product Quality","Have roles and responsibilities in-process and final inspections been identified?","Purpose: \nTo use demonstrated results from a LRIP line to verify adequate supplier controls and capability exists to meet FRP requirements. To ensure that continued audits and reviews are performed on suppliers as appropriate. The program manager must verify capability and controls from suppliers by using actual build data from LRIP. This early production data will lay the foundation for future production efforts. Management emphasis must be placed at suppliers where necessary to ensure production objectives can be achieved.","Additional Considerations: \n• Are supplier measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Is supplier non-conforming material is properly identified and corrective actions taken?\n• Supplier inspection status is clearly identified.\n• Supplier product quality records are available and meet appropriate requirements.","Lessons Learned: \n• A low-rate initial production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. Supplier quality systems must be validated to ensure they are using production processes.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[389,5,"F - Quality Management","F.2 - Product Quality","Have roles and responsibilities for statistical process controls for prototype units been identified?","Purpose: \nTo use demonstrated results from a LRIP line to verify adequate supplier controls and capability exists to meet FRP requirements. To ensure that continued audits and reviews are performed on suppliers as appropriate. The program manager must verify capability and controls from suppliers by using actual build data from LRIP. This early production data will lay the foundation for future production efforts. Management emphasis must be placed at suppliers where necessary to ensure production objectives can be achieved.","Additional Considerations: \n• Are supplier measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Is supplier non-conforming material is properly identified and corrective actions taken?\n• Supplier inspection status is clearly identified.\n• Supplier product quality records are available and meet appropriate requirements.","Lessons Learned: \n• A low-rate initial production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items. Supplier quality systems must be validated to ensure they are using production processes.\n• Statistical Process Control is not a requirement for all programs. Rates and quantities play a major role in the decision to manage product quality using SPC.\n• There should be a clear path of requirements flow down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[390,6,"F - Quality Management","F.2 - Product Quality","Has an approach for management of Key Characteristics been defined?  ","Purpose: \nTo verify the supplier base has adequate controls and capability to achieve production rate requirements. The program manager must constantly verify the supplier base controls and capability by using actual build data from supplier production lines to ensure confidence in achieving production requirements.","Additional Considerations: \n• Are supplier measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Supplier non-conforming material is properly identified and corrective actions taken.\n• Supplier inspection status is clearly identified.\n• Supplier product quality records are available and meet appropriate requirements.\n• Is the supplier conducting continuous improvement activities?","Lessons Learned: \n• Historically a large percent of the cost and problems have come from suppliers not staying up to speed in this area. The supplier base will be a good area to look for cost reductions. \n• Many prime contractors and their subcontractors will focus their efforts on continuous process improvement if it is a priority of the program office.\n• A full-rate production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items\n• There should be a clear path of requirements flow-down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[391,6,"F - Quality Management","F.2 - Product Quality","Have initial acceptance test requirements and procedures for EMD units been identified?","Purpose: \nTo verify the supplier base has adequate controls and capability to achieve production rate requirements. The program manager must constantly verify the supplier base controls and capability by using actual build data from supplier production lines to ensure confidence in achieving production requirements.","Additional Considerations: \n• Are supplier measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Supplier non-conforming material is properly identified and corrective actions taken.\n• Supplier inspection status is clearly identified.\n• Supplier product quality records are available and meet appropriate requirements.\n• Is the supplier conducting continuous improvement activities?","Lessons Learned: \n• Historically a large percent of the cost and problems have come from suppliers not staying up to speed in this area. The supplier base will be a good area to look for cost reductions. \n• Many prime contractors and their subcontractors will focus their efforts on continuous process improvement if it is a priority of the program office.\n• A full-rate production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items\n• There should be a clear path of requirements flow-down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[392,6,"F - Quality Management","F.2 - Product Quality","Have roles and responsibilities in-process and final inspections been identified?","Purpose: \nTo verify the supplier base has adequate controls and capability to achieve production rate requirements. The program manager must constantly verify the supplier base controls and capability by using actual build data from supplier production lines to ensure confidence in achieving production requirements.","Additional Considerations: \n• Are supplier measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?\n• Supplier non-conforming material is properly identified and corrective actions taken.\n• Supplier inspection status is clearly identified.\n• Supplier product quality records are available and meet appropriate requirements.\n• Is the supplier conducting continuous improvement activities?","Lessons Learned: \n• Historically a large percent of the cost and problems have come from suppliers not staying up to speed in this area. The supplier base will be a good area to look for cost reductions. \n• Many prime contractors and their subcontractors will focus their efforts on continuous process improvement if it is a priority of the program office.\n• A full-rate production environment is different for different products and is often dependent on rates and quantities of items being produced and the complexity of those items\n• There should be a clear path of requirements flow-down to suppliers from Key Performance Parameters (KPPs) to Key Characteristics and then to the use of SPC, if appropriate."]
[393,6,"F - Quality Management","F.2 - Product Quality","Have appropriate inspection and acceptance test procedures been identified for prototype units?","Purpose: \nTo evaluate whether new manufacturing skills will be needed to produce the alternatives being considered and if so, to assess the risk of obtaining required skills to build the products needed. Contractors need to establish very early whether new skills will be required and risk associated with obtaining those skills to meet program requirements. The skills might have to be developed prior to building test units in the earliest phase of a program, possibly before Milestone A.","Additional Considerations: \n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?","Lessons Learned: \n• When you are looking at skills, you must account for all the resources required to build, test/inspect and support (e.g. Manufacturing Engineers) the units."]
[394,7,"F - Quality Management","F.2 - Product Quality","Has quality data from the production representative environment been collected and analyzed? ","Purpose: \nTo perform a top level assessment for the program looking for areas of risk driven by workforce skills requirements. Where risks are identified, appropriate risk reduction efforts must be put into place in the TD Phase. In addition, cost should be reflected in both the estimates to execute the TD phase and overall lifecycle cost projections. The program must ensure the manufacturing workforce risk is assessed for each proposed alternative to make informed decisions on total risk of achieving program cost, schedule, and performance objectives.\n","Additional Considerations: \n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?","Lessons Learned: \n• When you are looking at skills, you must account for all resources required to build, test/inspect and support labor (e.g. Manufacturing Engineers) for the units. \n• If there is a new process, or tolerances are more difficult, there exists a stronger probability of new or critical skills required that may be difficult to obtain."]
[395,7,"F - Quality Management","F.2 - Product Quality","Have the results of the analysis of production representative quality data been used to shape improvement plans? ","Purpose: \nTo perform a top level assessment for the program looking for areas of risk driven by workforce skills requirements. Where risks are identified, appropriate risk reduction efforts must be put into place in the TD Phase. In addition, cost should be reflected in both the estimates to execute the TD phase and overall lifecycle cost projections. The program must ensure the manufacturing workforce risk is assessed for each proposed alternative to make informed decisions on total risk of achieving program cost, schedule, and performance objectives.\n","Additional Considerations: \n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?","Lessons Learned: \n• When you are looking at skills, you must account for all resources required to build, test/inspect and support labor (e.g. Manufacturing Engineers) for the units. \n• If there is a new process, or tolerances are more difficult, there exists a stronger probability of new or critical skills required that may be difficult to obtain."]
[396,7,"F - Quality Management","F.2 - Product Quality","Are control plans for management of Key Characteristics (KCs) complete?","Purpose: \nTo perform a top level assessment for the program looking for areas of risk driven by workforce skills requirements. Where risks are identified, appropriate risk reduction efforts must be put into place in the TD Phase. In addition, cost should be reflected in both the estimates to execute the TD phase and overall lifecycle cost projections. The program must ensure the manufacturing workforce risk is assessed for each proposed alternative to make informed decisions on total risk of achieving program cost, schedule, and performance objectives.\n","Additional Considerations: \n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?","Lessons Learned: \n• When you are looking at skills, you must account for all resources required to build, test/inspect and support labor (e.g. Manufacturing Engineers) for the units. \n• If there is a new process, or tolerances are more difficult, there exists a stronger probability of new or critical skills required that may be difficult to obtain."]
[397,7,"F - Quality Management","F.2 - Product Quality","Has development of Test and Inspection Plans for EMD units been initiated?","Purpose: \nTo reduce risk by producing prototype components in a production relevant environment by monitoring and assessing the workforce skills needed to produce prototypes and eventually production units. To begin identifying special skills needed for building prototype hardware and what special training, if any, is required. This effort will be used to ensure resources are available to produce prototype units in a relevant environment. The top level review can also be used to determine needed skills for future production and cost requirements.","Additional Considerations:\n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?","Lessons Learned: \n• Focus on skills required for new processes, materials or performance requirements either in building or inspection of the products. \n• If new or critical skills are required to produce the prototype units, be sure you have a good understanding of contractor plans for obtaining these resources. This area must be tracked closely because it is typically ignored, yet it requires the same level of attention as other critical areas."]
[398,8,"F - Quality Management","F.2 - Product Quality","Are Key Characteristics being actively managed? ","Purpose: \nTo reduce risk by producing prototype components in a production relevant environment by monitoring and assessing the workforce skills needed to produce prototypes and eventually production units. To begin identifying special skills needed for building prototype hardware and what special training, if any, is required. This effort will be used to ensure resources are available to produce prototype units in a relevant environment. The top level review can also be used to determine needed skills for future production and cost requirements.","Additional Considerations:\n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?","Lessons Learned: \n• Focus on skills required for new processes, materials or performance requirements either in building or inspection of the products. \n• If new or critical skills are required to produce the prototype units, be sure you have a good understanding of contractor plans for obtaining these resources. This area must be tracked closely because it is typically ignored, yet it requires the same level of attention as other critical areas."]
[399,8,"F - Quality Management","F.2 - Product Quality","Are measurement procedures and controls in place, such as Statistical Process Controls; a Failure, Reporting, Analysis, and Corrective Action System, quality audits, customer satisfaction monitoring, etc.? "]
[400,8,"F - Quality Management","F.2 - Product Quality","Are process capability requirements for all Key Characteristics being met by pilot line data?","Purpose: \nTo evaluate workforce skills by building prototypes in a production relevant environment. To assess and identify workforce skills needed to produce the development units for the EMD and eventually, Production. This is where you must demonstrate special skills to build prototype hardware and any special training required. This effort will be used to plan for resources to produce development units in a Production Representative and Pilot Production Line. o achieve requirements for pilot line and production?","Additional Considerations:\n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?","Lessons Learned: \n• You must work with the contractor/ supplier to identify what skills are required on a pilot line for your program. \n• Examine work effort that will migrate from labs and prototype areas to the factory floor, specifically new skills needed versus what are available.\n• Ensure plans address the skills needed by factory floor personnel are adequate to demonstrate a pilot line in the next phase. \n• One problem seen in entering EMD is that you may have been able to produce development units with highly skilled technicians, but your manpower skills are insufficient in a production environment."]
[401,8,"F - Quality Management","F.2 - Product Quality","Are all Test and Inspection Plans complete?","Purpose: \nTo evaluate workforce skills by building prototypes in a production relevant environment. To assess and identify workforce skills needed to produce the development units for the EMD and eventually, Production. This is where you must demonstrate special skills to build prototype hardware and any special training required. This effort will be used to plan for resources to produce development units in a Production Representative and Pilot Production Line. o achieve requirements for pilot line and production?","Additional Considerations:\n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?","Lessons Learned: \n• You must work with the contractor/ supplier to identify what skills are required on a pilot line for your program. \n• Examine work effort that will migrate from labs and prototype areas to the factory floor, specifically new skills needed versus what are available.\n• Ensure plans address the skills needed by factory floor personnel are adequate to demonstrate a pilot line in the next phase. \n• One problem seen in entering EMD is that you may have been able to produce development units with highly skilled technicians, but your manpower skills are insufficient in a production environment."]
[402,8,"F - Quality Management","F.2 - Product Quality","Have all Test and Inspection Plans for initial production (LRIP) units been validated?","Purpose: \nTo evaluate workforce skills by building prototypes in a production relevant environment. To assess and identify workforce skills needed to produce the development units for the EMD and eventually, Production. This is where you must demonstrate special skills to build prototype hardware and any special training required. This effort will be used to plan for resources to produce development units in a Production Representative and Pilot Production Line. o achieve requirements for pilot line and production?","Additional Considerations:\n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?","Lessons Learned: \n• You must work with the contractor/ supplier to identify what skills are required on a pilot line for your program. \n• Examine work effort that will migrate from labs and prototype areas to the factory floor, specifically new skills needed versus what are available.\n• Ensure plans address the skills needed by factory floor personnel are adequate to demonstrate a pilot line in the next phase. \n• One problem seen in entering EMD is that you may have been able to produce development units with highly skilled technicians, but your manpower skills are insufficient in a production environment."]
[403,9,"F - Quality Management","F.2 - Product Quality","Does analysis of Low Rate Initial Production (LRIP) quality data show that all Key Characteristics, as well as other critical-to-quality manufacturing processes are capable and under control for FRP?","Purpose: \nTo understand the risk of obtaining workforce skills to meet production requirements. To understand the magnitude of this risk, you must build development units in a production representative environment. This requires monitoring, assessing and identifying workforce skills needed to produce the development units for EMD and training the workforce for pilot line, and eventually, LRIP. Workforce skills requirements addressed during CDR as they relate to design requirements should be and reflected in cost and schedule projections for the Production phase. Here you will begin demonstrating special skills needed to build development unit hardware and what special training, if any, is needed for a production representative environment. This effort will be used to plan for resources required to produce development units in a Pilot Line.","Additional Considerations:\n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?","Lessons Learned: \n• You must work with the contractor/ supplier to identify what represents a pilot line for your program. \n• A key area to look at in this phase is work effort that will migrate from labs and prototype areas to the factory floor from TD to EMD, specifically what new skills are needed versus what are available.\n• Ensure plans address the skills needed by factory floor personnel are adequate to demonstrate a pilot line. \n• You must place special emphasis on getting factory personnel to produce the EMD hardware as much as possible and understand the risk as you migrate to production. If unique skills are required in production, ensure they are reflected in cost and schedule risk assessments. This early phase will focus more on the planning effort to migrate to the factory floor."]
[404,10,"F - Quality Management","F.2 - Product Quality","Are all Key Characteristics being managed and in control for FRP?","Purpose: \nTo understand the risk of obtaining workforce skills to meet production requirements. To understand the magnitude of this risk, you must build development units in a production representative environment. This requires monitoring, assessing and identifying workforce skills needed to produce the development units for EMD and training the workforce for pilot line, and eventually, LRIP. Workforce skills requirements addressed during CDR as they relate to design requirements should be and reflected in cost and schedule projections for the Production phase. Here you will begin demonstrating special skills needed to build development unit hardware and what special training, if any, is needed for a production representative environment. This effort will be used to plan for resources required to produce development units in a Pilot Line.","Additional Considerations:\n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?","Lessons Learned: \n• You must work with the contractor/ supplier to identify what represents a pilot line for your program. \n• A key area to look at in this phase is work effort that will migrate from labs and prototype areas to the factory floor from TD to EMD, specifically what new skills are needed versus what are available.\n• Ensure plans address the skills needed by factory floor personnel are adequate to demonstrate a pilot line. \n• You must place special emphasis on getting factory personnel to produce the EMD hardware as much as possible and understand the risk as you migrate to production. If unique skills are required in production, ensure they are reflected in cost and schedule risk assessments. This early phase will focus more on the planning effort to migrate to the factory floor."]
[405,10,"F - Quality Management","F.2 - Product Quality","Are all Key Characteristics meeting required targets?","Purpose: \nTo understand the risk of obtaining workforce skills to meet production requirements. To understand the magnitude of this risk, you must build development units in a production representative environment. This requires monitoring, assessing and identifying workforce skills needed to produce the development units for EMD and training the workforce for pilot line, and eventually, LRIP. Workforce skills requirements addressed during CDR as they relate to design requirements should be and reflected in cost and schedule projections for the Production phase. Here you will begin demonstrating special skills needed to build development unit hardware and what special training, if any, is needed for a production representative environment. This effort will be used to plan for resources required to produce development units in a Pilot Line.","Additional Considerations:\n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?","Lessons Learned: \n• You must work with the contractor/ supplier to identify what represents a pilot line for your program. \n• A key area to look at in this phase is work effort that will migrate from labs and prototype areas to the factory floor from TD to EMD, specifically what new skills are needed versus what are available.\n• Ensure plans address the skills needed by factory floor personnel are adequate to demonstrate a pilot line. \n• You must place special emphasis on getting factory personnel to produce the EMD hardware as much as possible and understand the risk as you migrate to production. If unique skills are required in production, ensure they are reflected in cost and schedule risk assessments. This early phase will focus more on the planning effort to migrate to the factory floor."]
[406,10,"F - Quality Management","F.2 - Product Quality","Does quality data show continuous improvement?","Purpose: \nTo understand the risk of obtaining workforce skills to meet production requirements. To understand the magnitude of this risk, you must build development units in a production representative environment. This requires monitoring, assessing and identifying workforce skills needed to produce the development units for EMD and training the workforce for pilot line, and eventually, LRIP. Workforce skills requirements addressed during CDR as they relate to design requirements should be and reflected in cost and schedule projections for the Production phase. Here you will begin demonstrating special skills needed to build development unit hardware and what special training, if any, is needed for a production representative environment. This effort will be used to plan for resources required to produce development units in a Pilot Line.","Additional Considerations:\n• Do existing personnel have similar skills to those required and/or can they be cost-effectively cross trained?","Lessons Learned: \n• You must work with the contractor/ supplier to identify what represents a pilot line for your program. \n• A key area to look at in this phase is work effort that will migrate from labs and prototype areas to the factory floor from TD to EMD, specifically what new skills are needed versus what are available.\n• Ensure plans address the skills needed by factory floor personnel are adequate to demonstrate a pilot line. \n• You must place special emphasis on getting factory personnel to produce the EMD hardware as much as possible and understand the risk as you migrate to production. If unique skills are required in production, ensure they are reflected in cost and schedule risk assessments. This early phase will focus more on the planning effort to migrate to the factory floor."]
[407,1,"F - Quality Management","F.3 - Supplier Quality Management","Have supplier quality and quality management systems state of the art been surveyed?","Purpose: \nTo demonstrate workforce skills required for production by producing development units in a pilot line environment, and to ensure the workforce has established an affordable and executable manufacturing process. This is where you demonstrate the skills needed for building production unit hardware. This information will be used to plan resources to make production units. The results of your pilot line will be used to assess the risk of achieving production projections on cost and schedule.","Additional Considerations:\n• Place special emphasis on any new skills being required on the factory floor.","Lessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier on what represents a pilot line for your programs. \n• You must place special emphasis on getting factory personnel to produce the EMD hardware as much as possible and understand the risk as you migrate to production. Ensure unique skills required in production are reflected in cost and schedule risk assessments."]
[408,2,"F - Quality Management","F.3 - Supplier Quality Management","Have initial supplier quality and quality management systems been evaluated?","Purpose: \nTo demonstrate workforce skills required for production by producing development units in a pilot line environment, and to ensure the workforce has established an affordable and executable manufacturing process. This is where you demonstrate the skills needed for building production unit hardware. This information will be used to plan resources to make production units. The results of your pilot line will be used to assess the risk of achieving production projections on cost and schedule.","Additional Considerations:\n• Place special emphasis on any new skills being required on the factory floor.","Lessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier on what represents a pilot line for your programs. \n• You must place special emphasis on getting factory personnel to produce the EMD hardware as much as possible and understand the risk as you migrate to production. Ensure unique skills required in production are reflected in cost and schedule risk assessments."]
[409,3,"F - Quality Management","F.3 - Supplier Quality Management","Have supplier quality and quality management system requirements for the system concept been identified?","Purpose: \nTo demonstrate workforce skills required for production by producing development units in a pilot line environment, and to ensure the workforce has established an affordable and executable manufacturing process. This is where you demonstrate the skills needed for building production unit hardware. This information will be used to plan resources to make production units. The results of your pilot line will be used to assess the risk of achieving production projections on cost and schedule.","Additional Considerations:\n• Place special emphasis on any new skills being required on the factory floor.","Lessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier on what represents a pilot line for your programs. \n• You must place special emphasis on getting factory personnel to produce the EMD hardware as much as possible and understand the risk as you migrate to production. Ensure unique skills required in production are reflected in cost and schedule risk assessments."]
[410,4,"F - Quality Management","F.3 - Supplier Quality Management","Have potential supplier quality capabilities, risks, and issues been identified for the preferred materiel solution, including subtier suppliers? ","Purpose: \nTo demonstrate workforce skills required for production by producing development units in a pilot line environment, and to ensure the workforce has established an affordable and executable manufacturing process. This is where you demonstrate the skills needed for building production unit hardware. This information will be used to plan resources to make production units. The results of your pilot line will be used to assess the risk of achieving production projections on cost and schedule.","Additional Considerations:\n• Place special emphasis on any new skills being required on the factory floor.","Lessons Learned: \n• One of the important steps for achieving this requirement is to work with the contractor/ supplier on what represents a pilot line for your programs. \n• You must place special emphasis on getting factory personnel to produce the EMD hardware as much as possible and understand the risk as you migrate to production. Ensure unique skills required in production are reflected in cost and schedule risk assessments."]
[411,4,"F - Quality Management","F.3 - Supplier Quality Management","Have supplier quality management system requirements been defined and documented in the AS? ","Purpose: \nTo demonstrate that manufacturing processes are under control and that workforce skills are available to achieve requirements (performance, schedule and cost objectives). To assess whether workforce planning has been adequately demonstrated/validated to allow for an orderly increase in production rate. This is where you will validate that planning has provided for the necessary special skills to ensure your processes are under control for building production units.","Additional Considerations:\n• Ensure production personnel are properly cross-trained for critical skills to avoid work stoppages in case of absences.","Lessons Learned: \n• As you transition to FRP, examine union agreements and other contingencies that might impact higher rates."]
[412,5,"F - Quality Management","F.3 - Supplier Quality Management","Have quality capabilities and risks from the supply chain (i.e., supplier base) been identified? ","Purpose: \nTo demonstrate that manufacturing processes are under control and that workforce skills are available to achieve requirements (performance, schedule and cost objectives). To assess whether workforce planning has been adequately demonstrated/validated to allow for an orderly increase in production rate. This is where you will validate that planning has provided for the necessary special skills to ensure your processes are under control for building production units.","Additional Considerations:\n• Ensure production personnel are properly cross-trained for critical skills to avoid work stoppages in case of absences.","Lessons Learned: \n• As you transition to FRP, examine union agreements and other contingencies that might impact higher rates."]
[413,5,"F - Quality Management","F.3 - Supplier Quality Management","Were the quality capabilities and risks from subtier supplier quality management also identified?","Purpose: \nTo demonstrate that the manufacturing processes are under control and that workforce skills are available to achieve Full Rate Production requirements (performance, schedule and cost objectives). To assess whether workforce planning has been adequately demonstrated/validated to allow for a possible surge in production rate.","Additional Considerations:\n• Are production workforce skill sets being maintained due to attrition of the workforce? \n• Are workforce personnel available, or can they be trained, to meet surge requirements if necessary?\n• Ensure production personnel are properly cross-trained for critical skills to avoid work stoppages in case of absences.","Lessons Learned: \n• One consideration as you migrate to higher rate production will be to examine union agreements and other contingencies that might impact higher rates.\n• Also look at the personnel demographics for key skills."]
[414,6,"F - Quality Management","F.3 - Supplier Quality Management","Have initiatives to address shortfalls in supplier Quality Management Systems been identified?","Purpose: \nTo demonstrate that the manufacturing processes are under control and that workforce skills are available to achieve Full Rate Production requirements (performance, schedule and cost objectives). To assess whether workforce planning has been adequately demonstrated/validated to allow for a possible surge in production rate.","Additional Considerations:\n• Are production workforce skill sets being maintained due to attrition of the workforce? \n• Are workforce personnel available, or can they be trained, to meet surge requirements if necessary?\n• Ensure production personnel are properly cross-trained for critical skills to avoid work stoppages in case of absences.","Lessons Learned: \n• One consideration as you migrate to higher rate production will be to examine union agreements and other contingencies that might impact higher rates.\n• Also look at the personnel demographics for key skills."]
[415,6,"F - Quality Management","F.3 - Supplier Quality Management","Have initiatives to address shortfalls in supply chain Quality Management been identified?","Purpose: \nTo evaluate whether new/unique tooling and test equipment will be required to produce proposed alternatives and to understand overall requirements of tooling and test equipment as it will impact the cost and performance risk. To begin risk planning. Contractors need to establish very early their tooling and test equipment concepts and whether they must develop new/unique tooling and test equipment to produce the proposed alternatives. If so, the risk must be addressed to achieving program requirements.","Additional Considerations: \n• None","Lessons Learned: \n• Tooling and test equipment can be major cost and risk drivers and must be addressed in the AoA. If new/unique tooling and test equipment is required, this must be tracked through EMD."]
[416,7,"F - Quality Management","F.3 - Supplier Quality Management","Do key supplier Quality Management Systems (QMSs) meet appropriate industry standards? ","Purpose: \nTo evaluate whether new/unique tooling and test equipment will be required to produce proposed alternatives and to understand overall requirements of tooling and test equipment as it will impact the cost and performance risk. To continue risk planning to address identified risks. Contractors need to establish tooling and test equipment concepts early in the program life-cycle and determine whether new/unique tooling and test equipment will be needed to produce the proposed alternative. If so, the risk must be addressed to achieving program requirements.","Additional Considerations: \n• None","Lessons Learned: \n• Tooling and test equipment can be major cost and risk drivers and must be addressed in the AoA. If new/unique tooling and test equipment is required, this must be tracked through EMD."]
[417,7,"F - Quality Management","F.3 - Supplier Quality Management","Has supplier quality data from production representative units been collected and analyzed?","Purpose: \nTo evaluate whether new/unique tooling and test equipment will be required to produce proposed alternatives and to understand overall requirements of tooling and test equipment as it will impact the cost and performance risk. To continue risk planning to address identified risks. Contractors need to establish tooling and test equipment concepts early in the program life-cycle and determine whether new/unique tooling and test equipment will be needed to produce the proposed alternative. If so, the risk must be addressed to achieving program requirements.","Additional Considerations: \n• None","Lessons Learned: \n• Tooling and test equipment can be major cost and risk drivers and must be addressed in the AoA. If new/unique tooling and test equipment is required, this must be tracked through EMD."]
[418,7,"F - Quality Management","F.3 - Supplier Quality Management","Has a strategy for audits of critical supplier processes been outlined?"]
[419,8,"F - Quality Management","F.3 - Supplier Quality Management","Are supplier program-specific Quality Management Systems (QMSs) adequate (i.e., meet all quality requirements)?","Purpose: \nTo understand what new/unique tooling and test equipment is required for a program by building prototype units in a production relevant environment. To determine the impact this equipment will have on program cost, schedule, and performance. All of this information is needed to understand cost and performance risk for the selected alternative and to begin risk management effort in the subsequent phases. Contractors need to establish tooling and test equipment concepts early in the program life-cycle and determine whether new/unique tooling and test equipment will be needed to produce the proposed alternative. If so, the risk must be addressed to achieving program requirements.","Additional Considerations: \n• None\n","Lessons Learned: \n• Tooling and test equipment can be major cost and risk drivers and must be addressed in the AoA. If new/unique tooling and test equipment are required it must be demonstrated in a production relevant environment. \n• It is very important that the customer work with the contractor to determine what defines a production relevant environment."]
[420,8,"F - Quality Management","F.3 - Supplier Quality Management","Is qualification testing for all products from suppliers completed?","Purpose: \nTo understand what new/unique tooling and test equipment is required for a program by building prototype units in a production relevant environment. To determine the impact this equipment will have on program cost, schedule, and performance. All of this information is needed to understand cost and performance risk for the selected alternative and to begin risk management effort in the subsequent phases. Contractors need to establish tooling and test equipment concepts early in the program life-cycle and determine whether new/unique tooling and test equipment will be needed to produce the proposed alternative. If so, the risk must be addressed to achieving program requirements.","Additional Considerations: \n• None\n","Lessons Learned: \n• Tooling and test equipment can be major cost and risk drivers and must be addressed in the AoA. If new/unique tooling and test equipment are required it must be demonstrated in a production relevant environment. \n• It is very important that the customer work with the contractor to determine what defines a production relevant environment."]
[421,8,"F - Quality Management","F.3 - Supplier Quality Management","Are all First Article Inspections (FAIs) for all products from suppliers completed?","Purpose: \nTo demonstrate the production tooling and test equipment by building early EMD units in a production representative environment. To ensure that production tooling and test equipment design and development is progressing so it can be used on the pilot line. To begin planning for overall maintenance requirements for tooling and test equipment. This is the beginning of the effort to understand manufacturing capability and risk in EMD. Contractors must demonstrate manufacturing capability in this phase, and tooling and test equipment is a key element of this activity. The development of required tooling and test equipment must be monitored closely, and using EMD builds can be a good approach to monitoring progress.","Additional Considerations: \n• Is there a documented record of calibration and certification of STE/SIE?\n","Lessons Learned: \n• The monitoring of the design, development and use of the tooling and test equipment needs to be accomplished because this equipment can be major cost and risk drivers. \n• Making sure the contractor has a maintenance strategy is important to ensure good product, and this area typically does not get addressed until there is a problem.\n• Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals."]
[422,8,"F - Quality Management","F.3 - Supplier Quality Management","Is acceptance testing of supplier products (i.e., processes and throughput) adequate to begin initial production (LRIP)?"]
[423,8,"F - Quality Management","F.3 - Supplier Quality Management","Are plans to conduct quality audits of subcontractor processes in place? ","Purpose: \nTo demonstrate the production tooling and test equipment by building early EMD units in a production representative environment. To ensure that production tooling and test equipment design and development is progressing so it can be used on the pilot line. To begin planning for overall maintenance requirements for tooling and test equipment. This is the beginning of the effort to understand manufacturing capability and risk in EMD. Contractors must demonstrate manufacturing capability in this phase, and tooling and test equipment is a key element of this activity. The development of required tooling and test equipment must be monitored closely, and using EMD builds can be a good approach to monitoring progress.","Additional Considerations: \n• Is there a documented record of calibration and certification of STE/SIE?\n","Lessons Learned: \n• The monitoring of the design, development and use of the tooling and test equipment needs to be accomplished because this equipment can be major cost and risk drivers. \n• Making sure the contractor has a maintenance strategy is important to ensure good product, and this area typically does not get addressed until there is a problem.\n• Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals."]
[424,8,"F - Quality Management","F.3 - Supplier Quality Management","Have quality audits of subcontractor processes been implemented?","Purpose: \nTo demonstrate that tooling and test equipment used on a pilot production line can meet program requirements. During this phase maintenance requirements on the tooling and test equipment must be demonstrated. All tooling and test equipment required for LRIP must be identified, and to the maximum extent practicable, demonstrated on the pilot line. If not, you must assess the risk of moving forward and ensure decision makers are aware of the risk and mitigation options.\nThe program must demonstrate manufacturing capability prior to proceeding into production and tooling and test equipment are an essential element for that demonstration.","\nAdditional Considerations: \n• Is there a documented record of calibration and certification of STE/SIE?","Lessons Learned: \n• Tooling and test equipment can be major cost and risk drivers, and their capability needs to be fully understood before proceeding into Production. If there will be tooling and test equipment used in production that have not yet demonstrated the required performance, you must mitigate the risk. \n• Examine the rate capability and utilization of EMD tooling and test equipment to ensure there is adequate capacity. If insufficient, ensure that the program has identified and budgeted money to obtain the required tooling and test equipment for LRIP. \n• In the past, program managers have assumed there was adequate test equipment capacity available using EMD assets, which turned out not to be case because of assets needed to support Operation Test and Evaluation (OT&E) or other test efforts. \n• Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals."]
[425,9,"F - Quality Management","F.3 - Supplier Quality Management","Does analysis of supplier quality data show that supplier quality management of Key Characteristics, as well as other critical-to-quality manufacturing processes are capable and under control ready for rate production (FRP)? ","Purpose: \nTo demonstrate that tooling and test equipment used on a pilot production line can meet program requirements. During this phase maintenance requirements on the tooling and test equipment must be demonstrated. All tooling and test equipment required for LRIP must be identified, and to the maximum extent practicable, demonstrated on the pilot line. If not, you must assess the risk of moving forward and ensure decision makers are aware of the risk and mitigation options.\nThe program must demonstrate manufacturing capability prior to proceeding into production and tooling and test equipment are an essential element for that demonstration.","\nAdditional Considerations: \n• Is there a documented record of calibration and certification of STE/SIE?","Lessons Learned: \n• Tooling and test equipment can be major cost and risk drivers, and their capability needs to be fully understood before proceeding into Production. If there will be tooling and test equipment used in production that have not yet demonstrated the required performance, you must mitigate the risk. \n• Examine the rate capability and utilization of EMD tooling and test equipment to ensure there is adequate capacity. If insufficient, ensure that the program has identified and budgeted money to obtain the required tooling and test equipment for LRIP. \n• In the past, program managers have assumed there was adequate test equipment capacity available using EMD assets, which turned out not to be case because of assets needed to support Operation Test and Evaluation (OT&E) or other test efforts. \n• Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals."]
[426,9,"F - Quality Management","F.3 - Supplier Quality Management","Is acceptance testing of supplier products (i.e., processes and throughput) adequate to begin FRP?"]
[427,9,"F - Quality Management","F.3 - Supplier Quality Management","Are quality audits of subcontractor processes to ensure subcontractor compliance to specifications being performed?","Purpose: \nTo demonstrate that tooling and test equipment used on a pilot production line can meet program requirements. During this phase maintenance requirements on the tooling and test equipment must be demonstrated. All tooling and test equipment required for LRIP must be identified, and to the maximum extent practicable, demonstrated on the pilot line. If not, you must assess the risk of moving forward and ensure decision makers are aware of the risk and mitigation options.\nThe program must demonstrate manufacturing capability prior to proceeding into production and tooling and test equipment are an essential element for that demonstration.","\nAdditional Considerations: \n• Is there a documented record of calibration and certification of STE/SIE?","Lessons Learned: \n• Tooling and test equipment can be major cost and risk drivers, and their capability needs to be fully understood before proceeding into Production. If there will be tooling and test equipment used in production that have not yet demonstrated the required performance, you must mitigate the risk. \n• Examine the rate capability and utilization of EMD tooling and test equipment to ensure there is adequate capacity. If insufficient, ensure that the program has identified and budgeted money to obtain the required tooling and test equipment for LRIP. \n• In the past, program managers have assumed there was adequate test equipment capacity available using EMD assets, which turned out not to be case because of assets needed to support Operation Test and Evaluation (OT&E) or other test efforts. \n• Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals."]
[428,10,"F - Quality Management","F.3 - Supplier Quality Management","Does analysis of supplier quality data show adequate management of Key Characteristics and control of critical manufacturing processes  (i.e., meeting Cpks), including supplier quality management of subtier suppliers?"]
[429,10,"F - Quality Management","F.3 - Supplier Quality Management","Do results of analysis of supplier quality data show a high statistical level (e.g. 6-sigma) on all critical dimensions?","Purpose: \nTo demonstrate that the tooling and test equipment have been adequately proven to proceed into Full Rate Production and that the maintenance schedule will support production requirements.\nThe program is required to demonstrate there is no significant manufacturing risk prior to proceeding to FRP. The capability and capacity of tooling and test equipment are an essential element in that demonstration.","Additional Considerations: \n• Is there a documented record of calibration and certification of STE/SIE?","Lessons Learned: \n• The capability and capacity of the tooling and test equipment can be major cost and risk drivers and must be addressed when you are demonstrating that there is no significant manufacturing risk for Full Rate Production. \n• Look at key elements in this area (e.g., cycle time, yield, scrap and rework, etc.) when you assess the capability to achieve program requirements and determining capacity. You must look at all of these elements to determine rate requirements for tooling and test equipment.\n• Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals."]
[430,10,"F - Quality Management","F.3 - Supplier Quality Management","Are quality audits of subcontractor processes to ensure subcontractor compliance to specifications being performed?","Purpose: \nTo demonstrate that the tooling and test equipment have been adequately proven to proceed into Full Rate Production and that the maintenance schedule will support production requirements.\nThe program is required to demonstrate there is no significant manufacturing risk prior to proceeding to FRP. The capability and capacity of tooling and test equipment are an essential element in that demonstration.","Additional Considerations: \n• Is there a documented record of calibration and certification of STE/SIE?","Lessons Learned: \n• The capability and capacity of the tooling and test equipment can be major cost and risk drivers and must be addressed when you are demonstrating that there is no significant manufacturing risk for Full Rate Production. \n• Look at key elements in this area (e.g., cycle time, yield, scrap and rework, etc.) when you assess the capability to achieve program requirements and determining capacity. You must look at all of these elements to determine rate requirements for tooling and test equipment.\n• Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals."]
[431,1,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have workforce skill sets to support emerging trends in manufacturing and technology been surveyed?","Purpose: \nTo demonstrate that the tooling and test equipment have been adequately proven to proceed into Full Rate Production and that the maintenance schedule will support production requirements.\nThe program is required to demonstrate there is no significant manufacturing risk prior to proceeding to FRP. The capability and capacity of tooling and test equipment are an essential element in that demonstration.","Additional Considerations: \n• Is there a documented record of calibration and certification of STE/SIE?","Lessons Learned: \n• The capability and capacity of the tooling and test equipment can be major cost and risk drivers and must be addressed when you are demonstrating that there is no significant manufacturing risk for Full Rate Production. \n• Look at key elements in this area (e.g., cycle time, yield, scrap and rework, etc.) when you assess the capability to achieve program requirements and determining capacity. You must look at all of these elements to determine rate requirements for tooling and test equipment.\n• Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals."]
[432,2,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have workforce skill sets to support emerging trends in manufacturing and technology been evaluated?"]
[433,3,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have workforce skill set requirements for system concepts been identified?","Purpose: \nTo proceed to Full Rate Production (i.e., you have demonstrated the capability and capacity to support program requirements and that the maintenance schedule will support production. The program is required to demonstrate control of the manufacturing process, and the tooling and test equipment is a critical part of that demonstration.","Additional Considerations: \n• Are sufficient tooling and test equipment available, or can it be obtained, to meet surge requirements, if necessary?\n• Is there a documented record of calibration and certification of STE/SIE?\n","Lessons Learned: \n• The capability and capacity of the tooling and test equipment can be major cost and risk drivers and must be addressed when you are demonstrating that there is no significant manufacturing risk for Full Rate Production. \n• Look at key elements in this area (e.g., cycle time, yield, scrap and rework, etc.) when you assess the capability to achieve program requirements and determining capacity. You must look at all of these elements to determine rate requirements for tooling and test equipment.\n• Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals."]
[434,3,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have workforce skill set capability gaps been identified?","Purpose: \nTo proceed to Full Rate Production (i.e., you have demonstrated the capability and capacity to support program requirements and that the maintenance schedule will support production. The program is required to demonstrate control of the manufacturing process, and the tooling and test equipment is a critical part of that demonstration.","Additional Considerations: \n• Are sufficient tooling and test equipment available, or can it be obtained, to meet surge requirements, if necessary?\n• Is there a documented record of calibration and certification of STE/SIE?\n","Lessons Learned: \n• The capability and capacity of the tooling and test equipment can be major cost and risk drivers and must be addressed when you are demonstrating that there is no significant manufacturing risk for Full Rate Production. \n• Look at key elements in this area (e.g., cycle time, yield, scrap and rework, etc.) when you assess the capability to achieve program requirements and determining capacity. You must look at all of these elements to determine rate requirements for tooling and test equipment.\n• Metrology of tooling and STE/SIE is especially critical for precision hardware, since it must be recalibrated and recertified at regular intervals."]
[435,4,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have workforce skill set and production workforce requirements (technical and operational) for the preferred materiel solution been identified?"]
[436,4,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have workforce skill set and production workforce requirements (technical and operational) for the preferred materiel solution been considered in the AoA?","Purpose: \nTo evaluate whether new/specialized facilities will be required to produce proposed alternatives. To use this information to understand cost and performance risk of selecting the best alternative and to begin risk planning to address identified risks. The contractor must determine whether specialized facilities are needed to produce items being considered. If specialized facilities will be needed, you must begin addressing the risk to achieving program requirements","Additional Considerations: \n• None","Lessons Learned: \n• None"]
[437,4,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have workforce training and development requirements to close skill set gaps been defined?","Purpose: \nTo identify facilities that will be required to produce proposed alternatives and to identify the facility requirements needed to produce pilot line items in EMD. To use this information to understand cost and performance risk in selecting the best alternative and to begin planning to address identified risks. Contractors must determine what facilities are required to produce prototype hardware very early in the product life-cycle and also plan where they will produce the units later in production. The lead time to address facilities can be extremely long, and if you plan to use different facilities (e.g., one facility to fabricate prototypes and one for production) you must understand the risk early in the program and make the decision of which facility to select and modify.","Additional Considerations: \n• Is available capital sufficient for required facilities, test equipment and tooling?","Lessons Learned: \n• Due to cost and schedule risk in this area, focus on both near term facility needs to produce prototypes and long term needs for production units. The facilities used to produce the prototype units are sometimes different from those used for production, or required production facilities may be significantly larger. If either situation occurs, it requires risk assessment and management to minimize the impact. The biggest issues are cost and schedule. The contractor must ensure he has sufficient capital to acquire the new facilities and that he starts his effort in adequate lead time to ensure new facilities are available when needed. \n• When you move to a new facility you need to address lost learning and ensure engineering and manufacturing support is in place."]
[438,4,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Has the availability of the workforce for TMRR Phase been determined?"]
[439,5,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have the required skill sets to meet prototype and production been identified?","Purpose: \nTo evaluate the availability of the manufacturing facilities that will be required to produce the alternatives being considered. To use this information to understand the performance risk in selecting the best alternative and to begin risk planning to address those risk identified. There should be a discussion on how planned facilities to produce the prototype units will meet the requirement to build items in a production relevant environment. If required facilities are not readily available to produce the prototype units, this area must be closely monitored to ensure the TD schedule can be met.","Additional Considerations: \n• Have sufficient funds been budgeted for required facilities, test equipment and tooling?\n• It is very important to begin discussions in this phase to identify what is a production relevant environment for facilities.","Lessons Learned: \n• The facilities used for the prototype must be assessed from both the perspective of producing prototypes and how they will simulate a production relevant environment. \n• In the past is that we did not adequately understand the risk of transitioning from one facility (e.g., laboratory) to a production environment. There is no requirement that facilities being used in this phase be the same as production, but it is important to understand the risk of transitioning to a new facility."]
[440,5,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Are plans to meet prototype and production workforce requirements developed?","Purpose: \nTo evaluate the availability of the manufacturing facilities that will be required to produce the alternatives being considered. To use this information to understand the performance risk in selecting the best alternative and to begin risk planning to address those risk identified. There should be a discussion on how planned facilities to produce the prototype units will meet the requirement to build items in a production relevant environment. If required facilities are not readily available to produce the prototype units, this area must be closely monitored to ensure the TD schedule can be met.","Additional Considerations: \n• Have sufficient funds been budgeted for required facilities, test equipment and tooling?\n• It is very important to begin discussions in this phase to identify what is a production relevant environment for facilities.","Lessons Learned: \n• The facilities used for the prototype must be assessed from both the perspective of producing prototypes and how they will simulate a production relevant environment. \n• In the past is that we did not adequately understand the risk of transitioning from one facility (e.g., laboratory) to a production environment. There is no requirement that facilities being used in this phase be the same as production, but it is important to understand the risk of transitioning to a new facility."]
[441,5,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have requirements for special skills certification and training (i.e., included in workforce plans) been established? "]
[442,6,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Are the required manufacturing workforce skills available for production in a relevant environment?","Purpose: \nTo understand the planning needed for the right facilities to demonstrate pilot line production. This information is needed to determine risk of proceeding into EMD. Due to the typical lead time and cost associated with facilities, it is important to address these issues as soon as possible.","Additional Considerations: \n• Has the budget been approved for required facilities, test equipment and tooling?","Lessons Learned: \n• In the past we have not adequately assessed facility requirements for demonstrating production early enough in the acquisition process. This has resulted in extending the EMD phase and/or adding significant risk as we transition into production."]
[443,6,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have the manufacturing workforce resources required to achieve pilot line and production requirements (i.e., number of personnel and skill sets) been identified? ","Purpose: \nTo understand the planning needed for the right facilities to demonstrate pilot line production. This information is needed to determine risk of proceeding into EMD. Due to the typical lead time and cost associated with facilities, it is important to address these issues as soon as possible.","Additional Considerations: \n• Has the budget been approved for required facilities, test equipment and tooling?","Lessons Learned: \n• In the past we have not adequately assessed facility requirements for demonstrating production early enough in the acquisition process. This has resulted in extending the EMD phase and/or adding significant risk as we transition into production."]
[444,6,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have the plans to achieve the manufacturing workforce resources required to achieve pilot line and production requirements been developed? "]
[445,7,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have the manufacturing workforce resources required to achieve pilot line requirements (i.e., number of personnel and skill sets) been identified?","Purpose: \nTo demonstrate facilities are adequate by building the early EMD units in a production representative environment. To ensure the Low Rate Initial Production (LRIP) facility planning effort is progressing satisfactorily to support LRIP requirements. Contractors must demonstrate manufacturing capability in this phase, and facilities are part of that demonstration. Acquiring facilities may require significant time and money; therefore, this effort needs to be monitored closely focusing on production of EMD units and planning for LRIP.","Additional Considerations: \n• Have funds been programmed or expended for necessary facilities, equipment and tooling?","Lessons Learned: \n• Facilities can be a major cost and schedule driver, so they need to be addressed in assessment of manufacturing capability. The monitoring of the facilities planning and progress needed to support the LRIP requirements must be a focus area, especially if new facilities are required or plans are being made to move to a different facility in LRIP. \n• You should start using planned production facilities as soon as practical in EMD."]
[446,7,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have the plans to achieve the manufacturing workforce resources required to achieve pilot line requirements been developed? ","Purpose: \nTo demonstrate facilities are adequate by building the early EMD units in a production representative environment. To ensure the Low Rate Initial Production (LRIP) facility planning effort is progressing satisfactorily to support LRIP requirements. Contractors must demonstrate manufacturing capability in this phase, and facilities are part of that demonstration. Acquiring facilities may require significant time and money; therefore, this effort needs to be monitored closely focusing on production of EMD units and planning for LRIP.","Additional Considerations: \n• Have funds been programmed or expended for necessary facilities, equipment and tooling?","Lessons Learned: \n• Facilities can be a major cost and schedule driver, so they need to be addressed in assessment of manufacturing capability. The monitoring of the facilities planning and progress needed to support the LRIP requirements must be a focus area, especially if new facilities are required or plans are being made to move to a different facility in LRIP. \n• You should start using planned production facilities as soon as practical in EMD."]
[447,7,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have the plans to achieve the manufacturing workforce resources required to achieve initial production (LRIP) requirements been updated?"]
[448,7,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Has the pilot line workforce been trained in the production representative environment?"]
[449,8,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have the manufacturing workforce resources required to achieve initial production (LRIP) requirements (i.e., number of personnel and skill sets) been identified?","Purpose: \nTo demonstrate that the pilot line can achieve program LRIP requirements. To ensure all the facilities required for LRIP are identified and, to the maximum extent practicable, demonstrated in the pilot line. If all facilities planned for LRIP cannot be demonstrated, you must assess the risk of moving forward and ensure decision-makers are aware of the risk and mitigation options. Finally, you need to assess planning to have adequate facilities for Full Rate Production (FRP). \nThe program is required to demonstrate manufacturing capability prior to Milestone C. The facilities needed for LRIP must be considered in making that determination. Due to the typical lead times and cost associated with facilities, it is important to address the requirements for FRP before you exit EMD.","Additional Considerations: \n• New facility construction, if required, should be virtually complete at this time.","Lessons Learned: \n• New facilities can be major cost and risk drivers, and their capability must be fully understood before proceeding into Production. If there are facilities required in production that have not yet been demonstrated in EMD, you must ensure risk management is applied. \n• You also need to verify rate capability and utilization of EMD facilities to be sure there is adequate capacity in those facilities or that the program manager has identified and has capital to obtain additional facilities for LRIP."]
[450,8,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have the plans to achieve the manufacturing workforce resources required to achieve initial production (LRIP) requirements been developed?","Purpose: \nTo demonstrate that the pilot line can achieve program LRIP requirements. To ensure all the facilities required for LRIP are identified and, to the maximum extent practicable, demonstrated in the pilot line. If all facilities planned for LRIP cannot be demonstrated, you must assess the risk of moving forward and ensure decision-makers are aware of the risk and mitigation options. Finally, you need to assess planning to have adequate facilities for Full Rate Production (FRP). \nThe program is required to demonstrate manufacturing capability prior to Milestone C. The facilities needed for LRIP must be considered in making that determination. Due to the typical lead times and cost associated with facilities, it is important to address the requirements for FRP before you exit EMD.","Additional Considerations: \n• New facility construction, if required, should be virtually complete at this time.","Lessons Learned: \n• New facilities can be major cost and risk drivers, and their capability must be fully understood before proceeding into Production. If there are facilities required in production that have not yet been demonstrated in EMD, you must ensure risk management is applied. \n• You also need to verify rate capability and utilization of EMD facilities to be sure there is adequate capacity in those facilities or that the program manager has identified and has capital to obtain additional facilities for LRIP."]
[451,8,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have production personnel been trained on the pilot line where possible?","Purpose: \nTo demonstrate that the pilot line can achieve program LRIP requirements. To ensure all the facilities required for LRIP are identified and, to the maximum extent practicable, demonstrated in the pilot line. If all facilities planned for LRIP cannot be demonstrated, you must assess the risk of moving forward and ensure decision-makers are aware of the risk and mitigation options. Finally, you need to assess planning to have adequate facilities for Full Rate Production (FRP). \nThe program is required to demonstrate manufacturing capability prior to Milestone C. The facilities needed for LRIP must be considered in making that determination. Due to the typical lead times and cost associated with facilities, it is important to address the requirements for FRP before you exit EMD.","Additional Considerations: \n• New facility construction, if required, should be virtually complete at this time.","Lessons Learned: \n• New facilities can be major cost and risk drivers, and their capability must be fully understood before proceeding into Production. If there are facilities required in production that have not yet been demonstrated in EMD, you must ensure risk management is applied. \n• You also need to verify rate capability and utilization of EMD facilities to be sure there is adequate capacity in those facilities or that the program manager has identified and has capital to obtain additional facilities for LRIP."]
[452,8,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Based on pilot line results, have the plans to achieve the manufacturing workforce resources required for Full Rate Production (FRP) requirements been initiated? ","Purpose: \nTo demonstrate that the pilot line can achieve program LRIP requirements. To ensure all the facilities required for LRIP are identified and, to the maximum extent practicable, demonstrated in the pilot line. If all facilities planned for LRIP cannot be demonstrated, you must assess the risk of moving forward and ensure decision-makers are aware of the risk and mitigation options. Finally, you need to assess planning to have adequate facilities for Full Rate Production (FRP). \nThe program is required to demonstrate manufacturing capability prior to Milestone C. The facilities needed for LRIP must be considered in making that determination. Due to the typical lead times and cost associated with facilities, it is important to address the requirements for FRP before you exit EMD.","Additional Considerations: \n• New facility construction, if required, should be virtually complete at this time.","Lessons Learned: \n• New facilities can be major cost and risk drivers, and their capability must be fully understood before proceeding into Production. If there are facilities required in production that have not yet been demonstrated in EMD, you must ensure risk management is applied. \n• You also need to verify rate capability and utilization of EMD facilities to be sure there is adequate capacity in those facilities or that the program manager has identified and has capital to obtain additional facilities for LRIP."]
[453,9,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have all workforce requirements for Low Rate Initial Production (LRIP) been met? "]
[454,9,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have the plans to achieve the manufacturing workforce resources required for rate production (FRP) requirements been implemented?"]
[455,10,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","Have all workforce requirements for FRP been met?","Purpose: \nTo demonstrate there is no significant manufacturing risk prior to proceeding into FRP. The capability and capacity of your facilities are an essential element in making that determination.","Additional Considerations: \n• None","Lessons Learned: \n• The capability and capacity of the facilities can be major cost and risk drivers and must be assessed if you wish to demonstrate that there is no significant manufacturing risk for Full Rate Production. You must examine the major elements that drive capacity (e.g., cycle time, yield, scrap and rework, etc.) when you assess capacity to achieve program requirements."]
[456,10,"G - Manufacturing Workforce","G.1 - Manufacturing Workforce","In spite of any workforce attrition, are required production workforce skill sets being maintained?","Purpose: \nTo demonstrate there is no significant manufacturing risk prior to proceeding into FRP. The capability and capacity of your facilities are an essential element in making that determination.","Additional Considerations: \n• None","Lessons Learned: \n• The capability and capacity of the facilities can be major cost and risk drivers and must be assessed if you wish to demonstrate that there is no significant manufacturing risk for Full Rate Production. You must examine the major elements that drive capacity (e.g., cycle time, yield, scrap and rework, etc.) when you assess capacity to achieve program requirements."]
[457,1,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Has state of the art tooling, test and inspection equipment been surveyed?"]
[458,2,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have potential tooling, STE, and SIE requirements been identified?"]
[459,3,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have tooling, STE, and SIE requirements for system concepts been identified?","Purpose: \nTo demonstrate control of the manufacturing process and the facilities used for FRP. The capability and capacity of your facilities, as it relates to your program requirements, are an essential element in making that determination.","Additional Considerations: \n• Is sufficient facility capacity available, or can it be obtained, to meet surge requirements, if necessary?","Lessons Learned: \n• The capability and capacity of the facilities can be major cost and risk drivers and need to be addressed when you are demonstrating that that manufacturing process are under control for Full Rate Production. You need to look at key elements in this area (e.g., cycle time, yield, scrap and rework, etc.) when you assess the capability and capacity."]
[460,3,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have special tooling, STE, and SIE gaps been identified?","Purpose: \nTo demonstrate control of the manufacturing process and the facilities used for FRP. The capability and capacity of your facilities, as it relates to your program requirements, are an essential element in making that determination.","Additional Considerations: \n• Is sufficient facility capacity available, or can it be obtained, to meet surge requirements, if necessary?","Lessons Learned: \n• The capability and capacity of the facilities can be major cost and risk drivers and need to be addressed when you are demonstrating that that manufacturing process are under control for Full Rate Production. You need to look at key elements in this area (e.g., cycle time, yield, scrap and rework, etc.) when you assess the capability and capacity."]
[461,4,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have tooling/Special Test Equipment (STE)/Special Inspection Equipment (SIE) requirements for the preferred materiel solution been considered as part of AoA?"]
[462,5,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have the requirements for tooling, Special Test Equipment, and/or Special Inspection Equipment been identified?"]
[463,5,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Do the identified requirements for tooling, Special Test Equipment, and/or Special Inspection Equipment include a supporting rationale?","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity. DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in a program’s life cycle is a root cause of cost and schedule growth. There is near universal agreement that examining manufacturing maturity for the first time just prior to production is a mistake; manufacturing readiness must be addressed early in the acquisition process and some phases of Science and Technology (S&T).","Additional Considerations: \n• Has a make-or-buy plan been developed for critical items at this stage?","Lessons Learned: \n• During the MSA Phase, an assessment of manufacturing feasibility should be conducted for each competing materiel solution being examined in the AoA. \n• The manufacturing risk of each proposed materiel solution should be assessed to determine whether it can achieve program cost, schedule, and performance objectives. The program should then begin addressing and mitigating any identified risk."]
[464,5,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Has a schedule for required tooling, Special Test Equipment, and/or Special Inspection Equipment been developed?","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity. DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in a program’s life cycle is a root cause of cost and schedule growth. There is near universal agreement that examining manufacturing maturity for the first time just prior to production is a mistake; manufacturing readiness must be addressed early in the acquisition process and some phases of Science and Technology (S&T).","Additional Considerations: \n• Has a make-or-buy plan been developed for critical items at this stage?","Lessons Learned: \n• During the MSA Phase, an assessment of manufacturing feasibility should be conducted for each competing materiel solution being examined in the AoA. \n• The manufacturing risk of each proposed materiel solution should be assessed to determine whether it can achieve program cost, schedule, and performance objectives. The program should then begin addressing and mitigating any identified risk."]
[465,6,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have prototype tooling, Special Test Equipment, and/or Special Inspection Equipment concepts been demonstrated in production relevant environment?  ","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity. To identify and assess the manufacturing risk of the alternatives being considered and to begin maturing manufacturing processes to reduce risk. One of the major activities in maturing processes in TD is to assess the degree to which the system being considered has been adequately demonstrated in a production relevant environment. DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in a program’s life cycle is a root cause of cost and schedule growth. Manufacturing readiness must be addressed early in the acquisition process and some phases of Science and Technology (S&T).","Additional Considerations:\n• A make-or-buy plan should have been developed for critical items at this stage.","Lessons Learned:\n• The MRL process is a best practice to capture and manage manufacturing risk before proceeding to EMD. While it is unlikely that contractors would have a complete factory and supply chain established this early in a program, there is still a need to obtain knowledge of critical manufacturing processes, production scale-up efforts, potential supply chain issues, and manufacturing maturity issues. \n• It is essential that a good baseline of the manufacturing maturity and risk be established at the beginning of TD in order to employ correct risk mitigation efforts. If your program went through MSA, the results would be a good start in addressing the manufacturing risk; nevertheless, it is recommended that a manufacturing assessment be conducted early TD to establish a baseline for manufacturing maturity and to determine manufacturing risk. \n• A major goal in TD is to mature the manufacturing process, but even more important, to identify, assess, and mitigate manufacturing risk before proceeding into EMD."]
[466,6,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Are efforts to develop requirements for production tooling, Special Test Equipment, and/or Special Inspection Equipment complete?  ","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity. To identify and assess the manufacturing risk of the alternatives being considered and to begin maturing manufacturing processes to reduce risk. One of the major activities in maturing processes in TD is to assess the degree to which the system being considered has been adequately demonstrated in a production relevant environment. DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in a program’s life cycle is a root cause of cost and schedule growth. Manufacturing readiness must be addressed early in the acquisition process and some phases of Science and Technology (S&T).","Additional Considerations:\n• A make-or-buy plan should have been developed for critical items at this stage.","Lessons Learned:\n• The MRL process is a best practice to capture and manage manufacturing risk before proceeding to EMD. While it is unlikely that contractors would have a complete factory and supply chain established this early in a program, there is still a need to obtain knowledge of critical manufacturing processes, production scale-up efforts, potential supply chain issues, and manufacturing maturity issues. \n• It is essential that a good baseline of the manufacturing maturity and risk be established at the beginning of TD in order to employ correct risk mitigation efforts. If your program went through MSA, the results would be a good start in addressing the manufacturing risk; nevertheless, it is recommended that a manufacturing assessment be conducted early TD to establish a baseline for manufacturing maturity and to determine manufacturing risk. \n• A major goal in TD is to mature the manufacturing process, but even more important, to identify, assess, and mitigate manufacturing risk before proceeding into EMD."]
[467,7,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have design and development efforts for production tooling, Special Test Equipment, and/or Special Inspection Equipment been initiated?","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity. To identify and assess the manufacturing risk of the alternatives being considered and to begin maturing the manufacturing processes to reduce risk. One of the major activities in maturing processes in TD is to assess the degree to which the system being considered has been adequately demonstrated in a production relevant environment.","Additional Considerations:\n• A make-or-buy plan should be developed for all BOM items at this stage.","Lessons Learned:\n• The MRL process is a best practice to capture and manage manufacturing risk before proceeding to EMD. While it is unlikely that contractors would have a complete factory and supply chain established this early in a program, there is still a need to obtain knowledge of critical manufacturing processes, production scale-up efforts, potential supply chain issues, and manufacturing maturity issues. \n• It is essential that a good baseline of the manufacturing maturity and risk be established at the beginning of TD in order to employ the correct risk mitigation efforts. If your program had an MSA phase, the results from that would be a good start in addressing the manufacturing risk, but it is recommended in any case that you conduct a manufacturing assessment early in TD to establish a baseline for manufacturing maturity and determine the manufacturing risk. \n• A major goal in this phase is to mature the manufacturing process, but even more important, to identify, assess, and mitigate manufacturing risk before proceeding to EMD."]
[468,7,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Are Special Test Equipment, and/or Special Inspection Equipment validation plans complete?","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity. To identify and assess the manufacturing risk of the alternatives being considered and to begin maturing the manufacturing processes to reduce risk. One of the major activities in maturing processes in TD is to assess the degree to which the system being considered has been adequately demonstrated in a production relevant environment.","Additional Considerations:\n• A make-or-buy plan should be developed for all BOM items at this stage.","Lessons Learned:\n• The MRL process is a best practice to capture and manage manufacturing risk before proceeding to EMD. While it is unlikely that contractors would have a complete factory and supply chain established this early in a program, there is still a need to obtain knowledge of critical manufacturing processes, production scale-up efforts, potential supply chain issues, and manufacturing maturity issues. \n• It is essential that a good baseline of the manufacturing maturity and risk be established at the beginning of TD in order to employ the correct risk mitigation efforts. If your program had an MSA phase, the results from that would be a good start in addressing the manufacturing risk, but it is recommended in any case that you conduct a manufacturing assessment early in TD to establish a baseline for manufacturing maturity and determine the manufacturing risk. \n• A major goal in this phase is to mature the manufacturing process, but even more important, to identify, assess, and mitigate manufacturing risk before proceeding to EMD."]
[469,7,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Has a manufacturing equipment (including tooling, Special Test Equipment, and/or Special Inspection Equipment) maintenance strategy been developed? ","Purpose: \nTo ensure there is a well-documented roadmap to achieve manufacturing maturity. To identify and assess the manufacturing risk of the alternatives being considered and to begin maturing the manufacturing processes to reduce risk. One of the major activities in maturing processes in TD is to assess the degree to which the system being considered has been adequately demonstrated in a production relevant environment.","Additional Considerations:\n• A make-or-buy plan should be developed for all BOM items at this stage.","Lessons Learned:\n• The MRL process is a best practice to capture and manage manufacturing risk before proceeding to EMD. While it is unlikely that contractors would have a complete factory and supply chain established this early in a program, there is still a need to obtain knowledge of critical manufacturing processes, production scale-up efforts, potential supply chain issues, and manufacturing maturity issues. \n• It is essential that a good baseline of the manufacturing maturity and risk be established at the beginning of TD in order to employ the correct risk mitigation efforts. If your program had an MSA phase, the results from that would be a good start in addressing the manufacturing risk, but it is recommended in any case that you conduct a manufacturing assessment early in TD to establish a baseline for manufacturing maturity and determine the manufacturing risk. \n• A major goal in this phase is to mature the manufacturing process, but even more important, to identify, assess, and mitigate manufacturing risk before proceeding to EMD."]
[470,8,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have all tooling, test, and inspection equipment been proven on the pilot line?","Purpose: \nTo build qualification test hardware on a Pilot Line to validate the manufacturing processes associated with the design. The initial manufacturing planning to accomplish this should be evident. Proper manufacturing planning and scheduling is necessary to accomplish these requirements. The need for a Manufacturing Plan is based on the Best Practices approach identified in the Willoughby Templates (DOD 4245.7-M) and the NAVSO P-6071. These documents explain that the Manufacturing Plan shall be sufficiently comprehensive so there is a high degree of confidence that the contractor has adequately evaluated and planned for production. The Manufacturing Plan should be used to monitor the contractual effort to ensure the timely and effective execution of the production program. These documents also stress the importance of managing the manufacturing approach of major subcontractors/suppliers to the success of the overall system.","Additional Considerations (Areas to Address):\n• Manufacturing Plan – The Manufacturing Plan documents the approach for fabricating a product in a cost-effective manner. It is based on the results of detailed planning and analyses conducted to define the optimum approach for product manufacture. All actions required to produce, test, and deliver acceptable systems on schedule at minimum cost should be documented in the plan. The Manufacturing plan should address aspects such as process flows, scheduling, tooling, test equipment, plant facilities, material decisions, manpower, manufacturing risk management, producibility, etc. A draft Manufacturing Plan should be delivered to the Government at the CDR.\n• Scheduling – The manufacturing schedule should provide assurance that the necessary resources will be available when needed, that no resources will be overloaded or expended during execution of any manufacturing tasks, and that product delivery dates are indeed achievable. The planning needed to complete the Pilot Line should be included in the Integrated Master Schedule (IMS).\n• Manufacturing Flow – Preliminary assembly and test flows should be developed to show how the product will move through the manufacturing plant from the stock room to the shipping deck. Activities such as Value Stream Mapping are extremely valuable in developing an efficient flow. This activity is useful in determining the need for resources and in determining if the manufacturing schedule is achievable.\n• Manufacturing Documentation – Adequate manufacturing documentation is necessary to ensure processes, assembly, and test operations are performed correctly. Manufacturing documentation includes items such as work instructions, test procedures, inspection procedures, kitting procedures, routers, travelers, tooling & test equipment lists, etc. Preliminary documentation should be completed such that Pilot line build can be effectively accomplished. \n• Manufacturing Risks – Manufacturing risks should have been identified and fed into the overall system risk management process. Proposed solutions should be verified in the Pilot Line build and system qualification tests.","Lessons Learned:\n• While a formal Manufacturing Plan is usually required as a contract deliverable item at the end of EMD, be careful not to wait until the end of EMD to address this area. To be effective, the manufacturing planning must be initiated at the beginning of EMD and proceed concurrently with the product design activities. The contract deliverable requirements should require a draft Manufacturing Plan to be delivered at the CDR. \n• Due to the fact that many prime contractors receive major components and assemblies from subcontractors, it is imperative that the requirements for proper manufacturing planning and scheduling be flowed down to subcontractors."]
[471,8,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have additional tooling, test and inspection equipment requirements been identified for Low Rate Initial Production (LRIP)? ","Purpose: \nTo build qualification test hardware on a Pilot Line to validate the manufacturing processes associated with the design. The initial manufacturing planning to accomplish this should be evident. Proper manufacturing planning and scheduling is necessary to accomplish these requirements. The need for a Manufacturing Plan is based on the Best Practices approach identified in the Willoughby Templates (DOD 4245.7-M) and the NAVSO P-6071. These documents explain that the Manufacturing Plan shall be sufficiently comprehensive so there is a high degree of confidence that the contractor has adequately evaluated and planned for production. The Manufacturing Plan should be used to monitor the contractual effort to ensure the timely and effective execution of the production program. These documents also stress the importance of managing the manufacturing approach of major subcontractors/suppliers to the success of the overall system.","Additional Considerations (Areas to Address):\n• Manufacturing Plan – The Manufacturing Plan documents the approach for fabricating a product in a cost-effective manner. It is based on the results of detailed planning and analyses conducted to define the optimum approach for product manufacture. All actions required to produce, test, and deliver acceptable systems on schedule at minimum cost should be documented in the plan. The Manufacturing plan should address aspects such as process flows, scheduling, tooling, test equipment, plant facilities, material decisions, manpower, manufacturing risk management, producibility, etc. A draft Manufacturing Plan should be delivered to the Government at the CDR.\n• Scheduling – The manufacturing schedule should provide assurance that the necessary resources will be available when needed, that no resources will be overloaded or expended during execution of any manufacturing tasks, and that product delivery dates are indeed achievable. The planning needed to complete the Pilot Line should be included in the Integrated Master Schedule (IMS).\n• Manufacturing Flow – Preliminary assembly and test flows should be developed to show how the product will move through the manufacturing plant from the stock room to the shipping deck. Activities such as Value Stream Mapping are extremely valuable in developing an efficient flow. This activity is useful in determining the need for resources and in determining if the manufacturing schedule is achievable.\n• Manufacturing Documentation – Adequate manufacturing documentation is necessary to ensure processes, assembly, and test operations are performed correctly. Manufacturing documentation includes items such as work instructions, test procedures, inspection procedures, kitting procedures, routers, travelers, tooling & test equipment lists, etc. Preliminary documentation should be completed such that Pilot line build can be effectively accomplished. \n• Manufacturing Risks – Manufacturing risks should have been identified and fed into the overall system risk management process. Proposed solutions should be verified in the Pilot Line build and system qualification tests.","Lessons Learned:\n• While a formal Manufacturing Plan is usually required as a contract deliverable item at the end of EMD, be careful not to wait until the end of EMD to address this area. To be effective, the manufacturing planning must be initiated at the beginning of EMD and proceed concurrently with the product design activities. The contract deliverable requirements should require a draft Manufacturing Plan to be delivered at the CDR. \n• Due to the fact that many prime contractors receive major components and assemblies from subcontractors, it is imperative that the requirements for proper manufacturing planning and scheduling be flowed down to subcontractors."]
[472,8,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have Special Test Equipment, and/or Special Inspection Equipment been validated as part of pilot line and IAW validation in accordance with the validation plan? ","Purpose: \nTo build qualification test hardware on a Pilot Line to validate the manufacturing processes associated with the design. The initial manufacturing planning to accomplish this should be evident. Proper manufacturing planning and scheduling is necessary to accomplish these requirements. The need for a Manufacturing Plan is based on the Best Practices approach identified in the Willoughby Templates (DOD 4245.7-M) and the NAVSO P-6071. These documents explain that the Manufacturing Plan shall be sufficiently comprehensive so there is a high degree of confidence that the contractor has adequately evaluated and planned for production. The Manufacturing Plan should be used to monitor the contractual effort to ensure the timely and effective execution of the production program. These documents also stress the importance of managing the manufacturing approach of major subcontractors/suppliers to the success of the overall system.","Additional Considerations (Areas to Address):\n• Manufacturing Plan – The Manufacturing Plan documents the approach for fabricating a product in a cost-effective manner. It is based on the results of detailed planning and analyses conducted to define the optimum approach for product manufacture. All actions required to produce, test, and deliver acceptable systems on schedule at minimum cost should be documented in the plan. The Manufacturing plan should address aspects such as process flows, scheduling, tooling, test equipment, plant facilities, material decisions, manpower, manufacturing risk management, producibility, etc. A draft Manufacturing Plan should be delivered to the Government at the CDR.\n• Scheduling – The manufacturing schedule should provide assurance that the necessary resources will be available when needed, that no resources will be overloaded or expended during execution of any manufacturing tasks, and that product delivery dates are indeed achievable. The planning needed to complete the Pilot Line should be included in the Integrated Master Schedule (IMS).\n• Manufacturing Flow – Preliminary assembly and test flows should be developed to show how the product will move through the manufacturing plant from the stock room to the shipping deck. Activities such as Value Stream Mapping are extremely valuable in developing an efficient flow. This activity is useful in determining the need for resources and in determining if the manufacturing schedule is achievable.\n• Manufacturing Documentation – Adequate manufacturing documentation is necessary to ensure processes, assembly, and test operations are performed correctly. Manufacturing documentation includes items such as work instructions, test procedures, inspection procedures, kitting procedures, routers, travelers, tooling & test equipment lists, etc. Preliminary documentation should be completed such that Pilot line build can be effectively accomplished. \n• Manufacturing Risks – Manufacturing risks should have been identified and fed into the overall system risk management process. Proposed solutions should be verified in the Pilot Line build and system qualification tests.","Lessons Learned:\n• While a formal Manufacturing Plan is usually required as a contract deliverable item at the end of EMD, be careful not to wait until the end of EMD to address this area. To be effective, the manufacturing planning must be initiated at the beginning of EMD and proceed concurrently with the product design activities. The contract deliverable requirements should require a draft Manufacturing Plan to be delivered at the CDR. \n• Due to the fact that many prime contractors receive major components and assemblies from subcontractors, it is imperative that the requirements for proper manufacturing planning and scheduling be flowed down to subcontractors."]
[473,8,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have maintenance process and procedures for manufacturing equipment (including tooling, Special Test Equipment, and/or Special Inspection Equipment) been demonstrated on pilot line?","Purpose: \nTo build qualification test hardware on a Pilot Line to validate the manufacturing processes associated with the design. The initial manufacturing planning to accomplish this should be evident. Proper manufacturing planning and scheduling is necessary to accomplish these requirements. The need for a Manufacturing Plan is based on the Best Practices approach identified in the Willoughby Templates (DOD 4245.7-M) and the NAVSO P-6071. These documents explain that the Manufacturing Plan shall be sufficiently comprehensive so there is a high degree of confidence that the contractor has adequately evaluated and planned for production. The Manufacturing Plan should be used to monitor the contractual effort to ensure the timely and effective execution of the production program. These documents also stress the importance of managing the manufacturing approach of major subcontractors/suppliers to the success of the overall system.","Additional Considerations (Areas to Address):\n• Manufacturing Plan – The Manufacturing Plan documents the approach for fabricating a product in a cost-effective manner. It is based on the results of detailed planning and analyses conducted to define the optimum approach for product manufacture. All actions required to produce, test, and deliver acceptable systems on schedule at minimum cost should be documented in the plan. The Manufacturing plan should address aspects such as process flows, scheduling, tooling, test equipment, plant facilities, material decisions, manpower, manufacturing risk management, producibility, etc. A draft Manufacturing Plan should be delivered to the Government at the CDR.\n• Scheduling – The manufacturing schedule should provide assurance that the necessary resources will be available when needed, that no resources will be overloaded or expended during execution of any manufacturing tasks, and that product delivery dates are indeed achievable. The planning needed to complete the Pilot Line should be included in the Integrated Master Schedule (IMS).\n• Manufacturing Flow – Preliminary assembly and test flows should be developed to show how the product will move through the manufacturing plant from the stock room to the shipping deck. Activities such as Value Stream Mapping are extremely valuable in developing an efficient flow. This activity is useful in determining the need for resources and in determining if the manufacturing schedule is achievable.\n• Manufacturing Documentation – Adequate manufacturing documentation is necessary to ensure processes, assembly, and test operations are performed correctly. Manufacturing documentation includes items such as work instructions, test procedures, inspection procedures, kitting procedures, routers, travelers, tooling & test equipment lists, etc. Preliminary documentation should be completed such that Pilot line build can be effectively accomplished. \n• Manufacturing Risks – Manufacturing risks should have been identified and fed into the overall system risk management process. Proposed solutions should be verified in the Pilot Line build and system qualification tests.","Lessons Learned:\n• While a formal Manufacturing Plan is usually required as a contract deliverable item at the end of EMD, be careful not to wait until the end of EMD to address this area. To be effective, the manufacturing planning must be initiated at the beginning of EMD and proceed concurrently with the product design activities. The contract deliverable requirements should require a draft Manufacturing Plan to be delivered at the CDR. \n• Due to the fact that many prime contractors receive major components and assemblies from subcontractors, it is imperative that the requirements for proper manufacturing planning and scheduling be flowed down to subcontractors."]
[474,8,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have all Special Test Equipment (STE) / Special Inspection equipment (SIE) been validated as part of pilot line validation In Accordance With (IAW) validation plan?","Purpose: \nTo build qualification test hardware on a Pilot Line to validate the manufacturing processes associated with the design. The initial manufacturing planning to accomplish this should be evident. Proper manufacturing planning and scheduling is necessary to accomplish these requirements. The need for a Manufacturing Plan is based on the Best Practices approach identified in the Willoughby Templates (DOD 4245.7-M) and the NAVSO P-6071. These documents explain that the Manufacturing Plan shall be sufficiently comprehensive so there is a high degree of confidence that the contractor has adequately evaluated and planned for production. The Manufacturing Plan should be used to monitor the contractual effort to ensure the timely and effective execution of the production program. These documents also stress the importance of managing the manufacturing approach of major subcontractors/suppliers to the success of the overall system.","Additional Considerations (Areas to Address):\n• Manufacturing Plan – The Manufacturing Plan documents the approach for fabricating a product in a cost-effective manner. It is based on the results of detailed planning and analyses conducted to define the optimum approach for product manufacture. All actions required to produce, test, and deliver acceptable systems on schedule at minimum cost should be documented in the plan. The Manufacturing plan should address aspects such as process flows, scheduling, tooling, test equipment, plant facilities, material decisions, manpower, manufacturing risk management, producibility, etc. A draft Manufacturing Plan should be delivered to the Government at the CDR.\n• Scheduling – The manufacturing schedule should provide assurance that the necessary resources will be available when needed, that no resources will be overloaded or expended during execution of any manufacturing tasks, and that product delivery dates are indeed achievable. The planning needed to complete the Pilot Line should be included in the Integrated Master Schedule (IMS).\n• Manufacturing Flow – Preliminary assembly and test flows should be developed to show how the product will move through the manufacturing plant from the stock room to the shipping deck. Activities such as Value Stream Mapping are extremely valuable in developing an efficient flow. This activity is useful in determining the need for resources and in determining if the manufacturing schedule is achievable.\n• Manufacturing Documentation – Adequate manufacturing documentation is necessary to ensure processes, assembly, and test operations are performed correctly. Manufacturing documentation includes items such as work instructions, test procedures, inspection procedures, kitting procedures, routers, travelers, tooling & test equipment lists, etc. Preliminary documentation should be completed such that Pilot line build can be effectively accomplished. \n• Manufacturing Risks – Manufacturing risks should have been identified and fed into the overall system risk management process. Proposed solutions should be verified in the Pilot Line build and system qualification tests.","Lessons Learned:\n• While a formal Manufacturing Plan is usually required as a contract deliverable item at the end of EMD, be careful not to wait until the end of EMD to address this area. To be effective, the manufacturing planning must be initiated at the beginning of EMD and proceed concurrently with the product design activities. The contract deliverable requirements should require a draft Manufacturing Plan to be delivered at the CDR. \n• Due to the fact that many prime contractors receive major components and assemblies from subcontractors, it is imperative that the requirements for proper manufacturing planning and scheduling be flowed down to subcontractors."]
[475,9,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Has all tooling, test, and inspection equipment (i.e., STE/SIE) been proven in the Low Rate Initial Production (LRIP)?","Purpose: \nTo ensure that an adequate manufacturing approach has been demonstrated in a low rate production environment and that adequate planning is available for cost-effective production at high rates. DoDI 5000.02 requires “demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes”.","Additional Considerations (See MRL 7 for additional areas to address): \n• Is the overall manufacturing approach adequate for FRP?\n• Have all manufacturing risks been identified with mitigation plan developed?\n• Are manufacturing risks incorporated into the overall risk management program?\n• Has a manufacturing flow been developed?\n• Has a Manufacturing Plan been developed to reflect the FRP build?\n• Have all Government furnished facilities, property, materials, and equipment been identified?\n• Have all work instructions, kitting procedures, shop floor travelers, etc. been developed?• Is this documentation under formal configuration control?\n• Has this documentation been demonstrated on the LRIP build?\n• Are work instructions available and/or posted on the factory floor at the point of use?\n• How frequently are work instructions reviewed and revised?\n• Are tooling designators clearly marked on the tooling to be used and referenced in the work instructions?\n• Do work instructions clearly and fully describe the assembly procedure?\n• Do work instructions contain visual aids?• Is this documentation under formal configuration control?\n• Has this documentation been demonstrated on the LRIP build?\n• Are work instructions available and/or posted on the factory floor at the point of use?\n• How frequently are work instructions reviewed and revised?\n• Are tooling designators clearly marked on the tooling to be used and referenced in the work instructions?\n• Do work instructions clearly and fully describe the assembly procedure?\n• Do work instructions contain visual aids?\n• Are all of the work in progress (WIP) and material used in assembly properly labeled so the status of each part and material can be readily ascertained? (Status would include which operations have been performed on the WIP and expiration data for shelf-life limited material.)\n• Is WIP properly protected against damage during the manufacturing process? \n• Is production material properly maintained against adverse effects of extreme heat or cold, moisture, ultraviolet light, unacceptable particulate matter (dust, dirt, grime, as applicable), or any other condition which would compromise the material?","Lessons Learned:\n• Increasing the production rate capability to FRP requirements effects many aspects of the manufacturing operations. Proactive planning is critical to ensure that efficient and economical production at these higher rates can be achieved. \n• It is also critical to fully understand the characteristics of the manufacturing processes so that proper controls can be implemented, and any risks associated with increasing the production rate can be identified and mitigated. \n• This proactive planning must also be flowed down to suppliers."]
[476,9,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have additional tooling, test, and inspection equipment (i.e., STE/SIE) requirements been identified for Full Rate Production (FRP)?","Purpose: \nTo ensure that an adequate manufacturing approach has been demonstrated in a low rate production environment and that adequate planning is available for cost-effective production at high rates. DoDI 5000.02 requires “demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes”.","Additional Considerations (See MRL 7 for additional areas to address): \n• Is the overall manufacturing approach adequate for FRP?\n• Have all manufacturing risks been identified with mitigation plan developed?\n• Are manufacturing risks incorporated into the overall risk management program?\n• Has a manufacturing flow been developed?\n• Has a Manufacturing Plan been developed to reflect the FRP build?\n• Have all Government furnished facilities, property, materials, and equipment been identified?\n• Have all work instructions, kitting procedures, shop floor travelers, etc. been developed?• Is this documentation under formal configuration control?\n• Has this documentation been demonstrated on the LRIP build?\n• Are work instructions available and/or posted on the factory floor at the point of use?\n• How frequently are work instructions reviewed and revised?\n• Are tooling designators clearly marked on the tooling to be used and referenced in the work instructions?\n• Do work instructions clearly and fully describe the assembly procedure?\n• Do work instructions contain visual aids?• Is this documentation under formal configuration control?\n• Has this documentation been demonstrated on the LRIP build?\n• Are work instructions available and/or posted on the factory floor at the point of use?\n• How frequently are work instructions reviewed and revised?\n• Are tooling designators clearly marked on the tooling to be used and referenced in the work instructions?\n• Do work instructions clearly and fully describe the assembly procedure?\n• Do work instructions contain visual aids?\n• Are all of the work in progress (WIP) and material used in assembly properly labeled so the status of each part and material can be readily ascertained? (Status would include which operations have been performed on the WIP and expiration data for shelf-life limited material.)\n• Is WIP properly protected against damage during the manufacturing process? \n• Is production material properly maintained against adverse effects of extreme heat or cold, moisture, ultraviolet light, unacceptable particulate matter (dust, dirt, grime, as applicable), or any other condition which would compromise the material?","Lessons Learned:\n• Increasing the production rate capability to FRP requirements effects many aspects of the manufacturing operations. Proactive planning is critical to ensure that efficient and economical production at these higher rates can be achieved. \n• It is also critical to fully understand the characteristics of the manufacturing processes so that proper controls can be implemented, and any risks associated with increasing the production rate can be identified and mitigated. \n• This proactive planning must also be flowed down to suppliers."]
[477,9,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Have maintenance process and procedures for manufacturing equipment (including tooling, Special Test Equipment, and/or Special Inspection Equipment) been demonstrated during Low Rate Initial Production (LRIP)?"]
[478,9,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Has validation of Special Test Equipment, and/or Special Inspection Equipment (STE/SIE) been maintained during Low Rate Initial Production (LRIP)?","Purpose: \nTo ensure that an adequate manufacturing approach has been demonstrated in a low rate production environment and that adequate planning is available for cost-effective production at high rates. DoDI 5000.02 requires “demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes”.","Additional Considerations (See MRL 7 for additional areas to address): \n• Is the overall manufacturing approach adequate for FRP?\n• Have all manufacturing risks been identified with mitigation plan developed?\n• Are manufacturing risks incorporated into the overall risk management program?\n• Has a manufacturing flow been developed?\n• Has a Manufacturing Plan been developed to reflect the FRP build?\n• Have all Government furnished facilities, property, materials, and equipment been identified?\n• Have all work instructions, kitting procedures, shop floor travelers, etc. been developed?• Is this documentation under formal configuration control?\n• Has this documentation been demonstrated on the LRIP build?\n• Are work instructions available and/or posted on the factory floor at the point of use?\n• How frequently are work instructions reviewed and revised?\n• Are tooling designators clearly marked on the tooling to be used and referenced in the work instructions?\n• Do work instructions clearly and fully describe the assembly procedure?\n• Do work instructions contain visual aids?• Is this documentation under formal configuration control?\n• Has this documentation been demonstrated on the LRIP build?\n• Are work instructions available and/or posted on the factory floor at the point of use?\n• How frequently are work instructions reviewed and revised?\n• Are tooling designators clearly marked on the tooling to be used and referenced in the work instructions?\n• Do work instructions clearly and fully describe the assembly procedure?\n• Do work instructions contain visual aids?\n• Are all of the work in progress (WIP) and material used in assembly properly labeled so the status of each part and material can be readily ascertained? (Status would include which operations have been performed on the WIP and expiration data for shelf-life limited material.)\n• Is WIP properly protected against damage during the manufacturing process? \n• Is production material properly maintained against adverse effects of extreme heat or cold, moisture, ultraviolet light, unacceptable particulate matter (dust, dirt, grime, as applicable), or any other condition which would compromise the material?","Lessons Learned:\n• Increasing the production rate capability to FRP requirements effects many aspects of the manufacturing operations. Proactive planning is critical to ensure that efficient and economical production at these higher rates can be achieved. \n• It is also critical to fully understand the characteristics of the manufacturing processes so that proper controls can be implemented, and any risks associated with increasing the production rate can be identified and mitigated. \n• This proactive planning must also be flowed down to suppliers."]
[479,10,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Is all tooling, test, and inspection equipment (i.e., STE/SIE) for Full Rate Production (FRP) in place? ","Purpose: \nTo ensure that an adequate manufacturing approach has been demonstrated in a low rate production environment and that adequate planning is available for cost-effective production at high rates. DoDI 5000.02 requires “demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes”.","Additional Considerations (See MRL 7 for additional areas to address): \n• Is the overall manufacturing approach adequate for FRP?\n• Have all manufacturing risks been identified with mitigation plan developed?\n• Are manufacturing risks incorporated into the overall risk management program?\n• Has a manufacturing flow been developed?\n• Has a Manufacturing Plan been developed to reflect the FRP build?\n• Have all Government furnished facilities, property, materials, and equipment been identified?\n• Have all work instructions, kitting procedures, shop floor travelers, etc. been developed?• Is this documentation under formal configuration control?\n• Has this documentation been demonstrated on the LRIP build?\n• Are work instructions available and/or posted on the factory floor at the point of use?\n• How frequently are work instructions reviewed and revised?\n• Are tooling designators clearly marked on the tooling to be used and referenced in the work instructions?\n• Do work instructions clearly and fully describe the assembly procedure?\n• Do work instructions contain visual aids?• Is this documentation under formal configuration control?\n• Has this documentation been demonstrated on the LRIP build?\n• Are work instructions available and/or posted on the factory floor at the point of use?\n• How frequently are work instructions reviewed and revised?\n• Are tooling designators clearly marked on the tooling to be used and referenced in the work instructions?\n• Do work instructions clearly and fully describe the assembly procedure?\n• Do work instructions contain visual aids?\n• Are all of the work in progress (WIP) and material used in assembly properly labeled so the status of each part and material can be readily ascertained? (Status would include which operations have been performed on the WIP and expiration data for shelf-life limited material.)\n• Is WIP properly protected against damage during the manufacturing process? \n• Is production material properly maintained against adverse effects of extreme heat or cold, moisture, ultraviolet light, unacceptable particulate matter (dust, dirt, grime, as applicable), or any other condition which would compromise the material?","Lessons Learned:\n• Increasing the production rate capability to FRP requirements effects many aspects of the manufacturing operations. Proactive planning is critical to ensure that efficient and economical production at these higher rates can be achieved. \n• It is also critical to fully understand the characteristics of the manufacturing processes so that proper controls can be implemented, and any risks associated with increasing the production rate can be identified and mitigated. \n• This proactive planning must also be flowed down to suppliers."]
[480,10,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Has a planned equipment maintenance schedule been achieved?","Purpose: \nTo ensure that an adequate manufacturing approach has been demonstrated in a low rate production environment and that adequate planning is available for cost-effective production at high rates. DoDI 5000.02 requires “demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes”.","Additional Considerations (See MRL 7 for additional areas to address): \n• Is the overall manufacturing approach adequate for FRP?\n• Have all manufacturing risks been identified with mitigation plan developed?\n• Are manufacturing risks incorporated into the overall risk management program?\n• Has a manufacturing flow been developed?\n• Has a Manufacturing Plan been developed to reflect the FRP build?\n• Have all Government furnished facilities, property, materials, and equipment been identified?\n• Have all work instructions, kitting procedures, shop floor travelers, etc. been developed?\n• Is this documentation under formal configuration control?• Has this documentation been demonstrated on the LRIP build?\n• Are work instructions available and/or posted on the factory floor at the point of use?\n• How frequently are work instructions reviewed and revised?\n• Are tooling designators clearly marked on the tooling to be used and referenced in the work instructions?\n• Do work instructions clearly and fully describe the assembly procedure?\n• Do work instructions contain visual aids?\n• Are all of the work in progress (WIP) and material used in assembly properly labeled so the status of each part and material can be readily ascertained? (Status would include which operations have been performed on the WIP and expiration data for shelf-life limited material.)\n• Is WIP properly protected against damage during the manufacturing process? \n• Is production material properly maintained against adverse effects of extreme heat or cold, moisture, ultraviolet light, unacceptable particulate matter (dust, dirt, grime, as applicable), or any other condition which would compromise the material?","Lessons Learned:\n• Increasing the production rate capability to FRP requirements effects many aspects of the manufacturing operations. Proactive planning is critical to ensure that efficient and economical production at these higher rates can be achieved. \n• It is also critical to fully understand the characteristics of the manufacturing processes so that proper controls can be implemented, and any risks associated with increasing the production rate can be identified and mitigated. \n• This proactive planning must also be flowed down to suppliers."]
[481,10,"H - Facilities","H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)","Has Special Test Equipment (STE) / Special Inspection equipment (SIE) validation been maintained as necessary?","Purpose: \nTo ensure that an adequate manufacturing approach has been demonstrated in a low rate production environment and that adequate planning is available for cost-effective production at high rates. DoDI 5000.02 requires “demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes”.","Additional Considerations (See MRL 7 for additional areas to address): \n• Is the overall manufacturing approach adequate for FRP?\n• Have all manufacturing risks been identified with mitigation plan developed?\n• Are manufacturing risks incorporated into the overall risk management program?\n• Has a manufacturing flow been developed?\n• Has a Manufacturing Plan been developed to reflect the FRP build?\n• Have all Government furnished facilities, property, materials, and equipment been identified?\n• Have all work instructions, kitting procedures, shop floor travelers, etc. been developed?\n• Is this documentation under formal configuration control?• Has this documentation been demonstrated on the LRIP build?\n• Are work instructions available and/or posted on the factory floor at the point of use?\n• How frequently are work instructions reviewed and revised?\n• Are tooling designators clearly marked on the tooling to be used and referenced in the work instructions?\n• Do work instructions clearly and fully describe the assembly procedure?\n• Do work instructions contain visual aids?\n• Are all of the work in progress (WIP) and material used in assembly properly labeled so the status of each part and material can be readily ascertained? (Status would include which operations have been performed on the WIP and expiration data for shelf-life limited material.)\n• Is WIP properly protected against damage during the manufacturing process? \n• Is production material properly maintained against adverse effects of extreme heat or cold, moisture, ultraviolet light, unacceptable particulate matter (dust, dirt, grime, as applicable), or any other condition which would compromise the material?","Lessons Learned:\n• Increasing the production rate capability to FRP requirements effects many aspects of the manufacturing operations. Proactive planning is critical to ensure that efficient and economical production at these higher rates can be achieved. \n• It is also critical to fully understand the characteristics of the manufacturing processes so that proper controls can be implemented, and any risks associated with increasing the production rate can be identified and mitigated. \n• This proactive planning must also be flowed down to suppliers."]
[482,1,"H - Facilities","H.2 - Facilities","Have current facility capabilities and capacity been surveyed?","Purpose: \nTo ensure that an adequate manufacturing approach has been demonstrated in a low rate production environment and that adequate planning is available for cost-effective production at high rates. DoDI 5000.02 requires “demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and the demonstrated control and capability of other critical processes”.","Additional Considerations (See MRL 7 for additional areas to address): \n• Is the overall manufacturing approach adequate for FRP?\n• Have all manufacturing risks been identified with mitigation plan developed?\n• Are manufacturing risks incorporated into the overall risk management program?\n• Has a manufacturing flow been developed?\n• Has a Manufacturing Plan been developed to reflect the FRP build?\n• Have all Government furnished facilities, property, materials, and equipment been identified?\n• Have all work instructions, kitting procedures, shop floor travelers, etc. been developed?\n• Is this documentation under formal configuration control?• Has this documentation been demonstrated on the LRIP build?\n• Are work instructions available and/or posted on the factory floor at the point of use?\n• How frequently are work instructions reviewed and revised?\n• Are tooling designators clearly marked on the tooling to be used and referenced in the work instructions?\n• Do work instructions clearly and fully describe the assembly procedure?\n• Do work instructions contain visual aids?\n• Are all of the work in progress (WIP) and material used in assembly properly labeled so the status of each part and material can be readily ascertained? (Status would include which operations have been performed on the WIP and expiration data for shelf-life limited material.)\n• Is WIP properly protected against damage during the manufacturing process? \n• Is production material properly maintained against adverse effects of extreme heat or cold, moisture, ultraviolet light, unacceptable particulate matter (dust, dirt, grime, as applicable), or any other condition which would compromise the material?","Lessons Learned:\n• Increasing the production rate capability to FRP requirements effects many aspects of the manufacturing operations. Proactive planning is critical to ensure that efficient and economical production at these higher rates can be achieved. \n• It is also critical to fully understand the characteristics of the manufacturing processes so that proper controls can be implemented, and any risks associated with increasing the production rate can be identified and mitigated. \n• This proactive planning must also be flowed down to suppliers."]
[483,2,"H - Facilities","H.2 - Facilities","Have potential facility capabilities and capacity requirements been identified?","Purpose: \nTo ensure that all manufacturing planning and scheduling activities have been demonstrated in a high rate production environment, and are sufficient for continued cost-effective production at high rates.","Additional Considerations:\n• Has manufacturing planning been made for surge and/or other contingency production?","Lessons Learned:\n• Increasing the production rate capability to FRP requirements effects many aspects of the manufacturing operations. Proactive planning is critical to ensure that efficient and economical production at these higher rates can be achieved. \n• It is also critical to fully understand the characteristics of the manufacturing processes so that proper controls can be implemented, and any risks associated with increasing the production rate can be identified and mitigated. \n• This proactive planning must also be flowed down to suppliers."]
[484,3,"H - Facilities","H.2 - Facilities","Have facility capabilities and capacity requirements and gaps for system concepts been identified? ","Purpose: \nTo ensure risks associated with material selection are being addressed. DOD case studies and GAO reports have demonstrated that the failure to adequately manage manufacturing risk early in a program’s life cycle is a root cause of cost and schedule growth. There is near universal agreement that examining manufacturing maturity for the first time just prior to production is a mistake; manufacturing readiness must be addressed early in the acquisition process and some phases of Science and Technology (S&T). Recognition of the need to address manufacturing readiness earlier in acquisition policy was implemented by Department of Defense Instruction 5000.02, Operation of the Defense Acquisition System, dated 8 December 2008. The policy established maturity criteria for measuring risks associated with manufacturing processes for Milestone A.","Additional Considerations: \n• None","Lessons Learned: \n• During the MSA Phase, an assessment of manufacturing feasibility should be conducted for each competing materiel solution being examined in the AoA. \n• The manufacturing risk of each proposed materiel solution should be assessed to determine whether it can achieve program cost, schedule, and performance objectives. The program should then begin addressing and mitigating any identified risk. \n• The list of material needed for Technology Development components should be developed, with associated lead time estimates."]
[485,4,"H - Facilities","H.2 - Facilities","Have capability and availability of mfg facilities for prototype development and production of the preferred materiel solution been evaluated? ","Purpose: \nTo ensure risks associated with material selection are being addressed. One of the key activities in maturing the manufacturing process in the TD phase is to assess the degree to which the materials being considered have been adequately demonstrated in a relevant environment. DOD case studies and GAO reports have demonstrated that failure to adequately manage manufacturing risk early in a program’s life cycle is a root cause of cost and schedule growth. There is near universal agreement that examining manufacturing maturity for the first time just prior to production is a mistake; manufacturing readiness must be addressed early in the acquisition process and some phases of Science and Technology (S&T). Recognition of the need to address manufacturing readiness earlier in acquisition policy was implemented by Department of Defense Instruction 5000.02, Operation of the Defense Acquisition System, dated 8 December 2008. The policy established maturity criteria for measuring risks associated with manufacturing processes for Milestones B.","\n\nAdditional Considerations:\n• Have you checked GIDEP alerts for obsolete or DMSMS materials?","Lessons Learned:\n• The material selection process should be ongoing. The parts list for TD units should be maturing. The Make/Buy evaluations should have begun. These evaluations should consider which manufacturing capabilities are available at the prime contractor and which ones will need to be farmed out to suppliers. \n• The Make/Buy selections should also reflect the proposed manufacturing approach for the pilot line, LRIP, and FRP. Lead times and other risks associated with material selection should be identified."]
[486,4,"H - Facilities","H.2 - Facilities","Have capability and availability of mfg facilities for prototype development and production of the preferred materiel solution been included in the AoA? ","Purpose: \nTo ensure risks associated with material selection are being addressed. One of the key activities in maturing the manufacturing process in the TD phase is to assess the degree to which the materials being considered have been adequately demonstrated in a relevant environment. DOD case studies and GAO reports have demonstrated that failure to adequately manage manufacturing risk early in a program’s life cycle is a root cause of cost and schedule growth. There is near universal agreement that examining manufacturing maturity for the first time just prior to production is a mistake; manufacturing readiness must be addressed early in the acquisition process and some phases of Science and Technology (S&T). Recognition of the need to address manufacturing readiness earlier in acquisition policy was implemented by Department of Defense Instruction 5000.02, Operation of the Defense Acquisition System, dated 8 December 2008. The policy established maturity criteria for measuring risks associated with manufacturing processes for Milestones B.","\n\nAdditional Considerations:\n• Have you checked GIDEP alerts for obsolete or DMSMS materials?","Lessons Learned:\n• The material selection process should be ongoing. The parts list for TD units should be maturing. The Make/Buy evaluations should have begun. These evaluations should consider which manufacturing capabilities are available at the prime contractor and which ones will need to be farmed out to suppliers. \n• The Make/Buy selections should also reflect the proposed manufacturing approach for the pilot line, LRIP, and FRP. Lead times and other risks associated with material selection should be identified."]
[487,4,"H - Facilities","H.2 - Facilities","Have capability and availability of mfg facilities for prototype development and production of the preferred materiel solution been documented in the AS and SEP? "]
[488,4,"H - Facilities","H.2 - Facilities","Has human factors & ergonomics and safety requirements for manufacturing (personnel, processes & equipment) been identified?","Purpose: \nTo ensure risks associated with material selection are being addressed. One of the key activities in maturing the manufacturing process in the TD phase is to assess the degree to which the materials being considered have been adequately demonstrated in a manufacturing relevant environment. DoDI 5000.02 requires “The project shall exit the Technology Development Phase when an affordable program or increment of militarily useful capability has been identified; the technology and manufacturing processes for that program or increment have been assessed and demonstrated in a relevant environment; manufacturing risks have been identified; a system or increment can be developed for production within a short timeframe (normally less than 5 years for weapon systems).”","Additional Considerations:\n• Have you checked GIDEP alerts for obsolete or DMSMS materials?\n","Lessons Learned:\n• The material selection process should be well underway with significant maturity. Most Make/Buy evaluations should be complete. These evaluations should consider which manufacturing capabilities are available at the prime contractor and which ones will need to be farmed out to suppliers. \n• The Make/Buy selections should also reflect the proposed manufacturing approach for the pilot line, LRIP, and FRP. Lead times and other risks associated with material selection should be identified. The development of the BOM should be initiated. \n• A major goal in this phase is to mature the material selection process, but even more important, to identify, assess, and mitigate manufacturing risk before proceeding to EMD.\n• It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete so they can avoid their use in new designs, if possible."]
[489,5,"H - Facilities","H.2 - Facilities","Have manufacturing facilities that meet requirements to produce prototypes been identified?","Purpose: \nTo ensure risks associated with material selection are being addressed. One of the key activities in maturing the manufacturing process in the TD phase is to assess the degree to which the materials being considered have been adequately demonstrated in a manufacturing relevant environment. DoDI 5000.02 requires “The project shall exit the Technology Development Phase when an affordable program or increment of militarily useful capability has been identified; the technology and manufacturing processes for that program or increment have been assessed and demonstrated in a relevant environment; manufacturing risks have been identified; a system or increment can be developed for production within a short timeframe (normally less than 5 years for weapon systems).”","Additional Considerations:\n• Have you checked GIDEP alerts for obsolete or DMSMS materials?\n","Lessons Learned:\n• The material selection process should be well underway with significant maturity. Most Make/Buy evaluations should be complete. These evaluations should consider which manufacturing capabilities are available at the prime contractor and which ones will need to be farmed out to suppliers. \n• The Make/Buy selections should also reflect the proposed manufacturing approach for the pilot line, LRIP, and FRP. Lead times and other risks associated with material selection should be identified. The development of the BOM should be initiated. \n• A major goal in this phase is to mature the material selection process, but even more important, to identify, assess, and mitigate manufacturing risk before proceeding to EMD.\n• It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete so they can avoid their use in new designs, if possible."]
[490,5,"H - Facilities","H.2 - Facilities","Have plans to produce prototypes in identified facilities been developed?","Purpose: \nTo ensure risks associated with material selection are being addressed. One of the key activities in maturing the manufacturing process in the TD phase is to assess the degree to which the materials being considered have been adequately demonstrated in a manufacturing relevant environment. DoDI 5000.02 requires “The project shall exit the Technology Development Phase when an affordable program or increment of militarily useful capability has been identified; the technology and manufacturing processes for that program or increment have been assessed and demonstrated in a relevant environment; manufacturing risks have been identified; a system or increment can be developed for production within a short timeframe (normally less than 5 years for weapon systems).”","Additional Considerations:\n• Have you checked GIDEP alerts for obsolete or DMSMS materials?\n","Lessons Learned:\n• The material selection process should be well underway with significant maturity. Most Make/Buy evaluations should be complete. These evaluations should consider which manufacturing capabilities are available at the prime contractor and which ones will need to be farmed out to suppliers. \n• The Make/Buy selections should also reflect the proposed manufacturing approach for the pilot line, LRIP, and FRP. Lead times and other risks associated with material selection should be identified. The development of the BOM should be initiated. \n• A major goal in this phase is to mature the material selection process, but even more important, to identify, assess, and mitigate manufacturing risk before proceeding to EMD.\n• It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete so they can avoid their use in new designs, if possible."]
[491,5,"H - Facilities","H.2 - Facilities","Have requirements for human factors and ergonomics for manufacturing personnel, processes, and equipment in identified facilities been assessed?","Purpose: \nTo ensure risks associated with material selection are being addressed. One of the key activities in maturing the manufacturing process in the TD phase is to assess the degree to which the materials being considered have been adequately demonstrated in a manufacturing relevant environment. DoDI 5000.02 requires “The project shall exit the Technology Development Phase when an affordable program or increment of militarily useful capability has been identified; the technology and manufacturing processes for that program or increment have been assessed and demonstrated in a relevant environment; manufacturing risks have been identified; a system or increment can be developed for production within a short timeframe (normally less than 5 years for weapon systems).”","Additional Considerations:\n• Have you checked GIDEP alerts for obsolete or DMSMS materials?\n","Lessons Learned:\n• The material selection process should be well underway with significant maturity. Most Make/Buy evaluations should be complete. These evaluations should consider which manufacturing capabilities are available at the prime contractor and which ones will need to be farmed out to suppliers. \n• The Make/Buy selections should also reflect the proposed manufacturing approach for the pilot line, LRIP, and FRP. Lead times and other risks associated with material selection should be identified. The development of the BOM should be initiated. \n• A major goal in this phase is to mature the material selection process, but even more important, to identify, assess, and mitigate manufacturing risk before proceeding to EMD.\n• It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete so they can avoid their use in new designs, if possible."]
[492,6,"H - Facilities","H.2 - Facilities","Have manufacturing facilities that meet requirements for pilot line been identified?","Purpose: \nTo build qualification test hardware on a Pilot Line using materials planned for LRIP. The material resource planning system should be validated on the Pilot Line build. MRLs play a vital role in ensuring that manufacturing processes have been effectively demonstrated in a pilot line environment and that industrial capabilities are reasonably available. In order to accomplish this, the material selection must be complete and stable, and an effective system to manage these materials needs to be established.","Additional Considerations: \n• What parts do we need to make or buy (Purchasing Plan)? \n• How many of these parts do we need (Capacity Plan)? \n• When must these parts be available (Detailed Manufacturing Schedule)? \n• Have you checked GIDEP alerts for obsolete or DMSMS materials?\n","Lessons Learned:\n• The more complex the manufacturing operation the more complex the MRP. MRP systems generate two basic outputs, the Purchasing Plan and Schedule that lays out when the purchase orders (POs) should be released and when the purchased items should be received in order to support the production dates. The second output is the Capacity Plan or Production Schedule. \n• The production schedule details the start and completion dates for steps in the production process (routing) to include how many items will be produced in each batch and what is required from the bill of materials to support the fabrication and assembly. The MRP system should be in place for the Pilot Line build.\n• The Make/Buy decisions and BOM should be complete for Pilot Line build. Changes to these at this phase in the acquisition cycle could introduce manufacturing risks. Stable decisions also assist in forecasting, impact analysis, component analysis, and other DMSMS-related activities. If the underlying information is even slightly off, then you will have problems with your MRP system. Past issues have been classified by the saying \"garbage in, garbage out.\" The quality of the following information sources needs to be correct to prevent errors in your MRP system: (1) Master Production System (MPS); (2) Bills of Material (BOM); (3) Inventory and Inventory Status; (4) Lead Times; (5) Production Size; (6) Production Schedule (working times); (7) Quality Data (yield data, scrap, rework and repair, etc.); (8) Safety Stock Levels and Times; and (9) Work-in-Progress Data. \n• It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete."]
[493,6,"H - Facilities","H.2 - Facilities","Have plans for building pilot line in identified facilities been developed?"]
[494,6,"H - Facilities","H.2 - Facilities","Have requirements for human factors and ergonomics for manufacturing personnel, processes, and equipment been verified in a production relevant environment?","Purpose: \nTo build qualification test hardware on a Pilot Line using materials planned for LRIP. The material resource planning system should be validated on the Pilot Line build. MRLs play a vital role in ensuring that manufacturing processes have been effectively demonstrated in a pilot line environment and that industrial capabilities are reasonably available. In order to accomplish this, the material selection must be complete and stable, and an effective system to manage these materials needs to be established.","Additional Considerations: \n• What parts do we need to make or buy (Purchasing Plan)? \n• How many of these parts do we need (Capacity Plan)? \n• When must these parts be available (Detailed Manufacturing Schedule)? \n• Have you checked GIDEP alerts for obsolete or DMSMS materials?\n","Lessons Learned:\n• The more complex the manufacturing operation the more complex the MRP. MRP systems generate two basic outputs, the Purchasing Plan and Schedule that lays out when the purchase orders (POs) should be released and when the purchased items should be received in order to support the production dates. The second output is the Capacity Plan or Production Schedule. \n• The production schedule details the start and completion dates for steps in the production process (routing) to include how many items will be produced in each batch and what is required from the bill of materials to support the fabrication and assembly. The MRP system should be in place for the Pilot Line build.\n• The Make/Buy decisions and BOM should be complete for Pilot Line build. Changes to these at this phase in the acquisition cycle could introduce manufacturing risks. Stable decisions also assist in forecasting, impact analysis, component analysis, and other DMSMS-related activities. If the underlying information is even slightly off, then you will have problems with your MRP system. Past issues have been classified by the saying \"garbage in, garbage out.\" The quality of the following information sources needs to be correct to prevent errors in your MRP system: (1) Master Production System (MPS); (2) Bills of Material (BOM); (3) Inventory and Inventory Status; (4) Lead Times; (5) Production Size; (6) Production Schedule (working times); (7) Quality Data (yield data, scrap, rework and repair, etc.); (8) Safety Stock Levels and Times; and (9) Work-in-Progress Data. \n• It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete."]
[495,7,"H - Facilities","H.2 - Facilities","Have manufacturing facilities been identified to produce the Low Rate Initial Production (LRIP) build?","Purpose: \nTo ensure that materials planning has been properly developed to allow for duplicating a product configuration in a cost-effective manner, and that any risks to achieving this goal have been identified and addressed. MRLs play a vital role in ensuring that manufacturing processes have been effectively demonstrated in a pilot line environment and that industrial capabilities are reasonably available. In order to accomplish this, the material selection must be complete and stable, and an effective system to manage these materials needs to be established.","Additional Considerations: \n• What parts do we need to make or buy (Purchasing Plan)? \n• How many of these parts do we need (Capacity Plan)? \n• When must these parts be available (Detailed Manufacturing Schedule)? \n• Have you continued to check GIDEP alerts for obsolete or DMSMS materials?\n• The more complex the manufacturing operation the more complex the MRP. MRP systems generate two basic outputs, the Purchasing Plan and Schedule that lays out when the purchase orders (POs) should be released and when the purchased items should be received in order to support the production dates. \n• The second output is the Capacity Plan or Production Schedule. The production schedule details the start and completion dates for steps in the production process (routing) to include how many items will be produced in each batch and what is required from the bill of materials to support the fabrication and assembly. \n• The MRP system should be in place for the Pilot Line build.","Lessons Learned:\n• The Make/Buy decisions and BOM should be complete. Changes to these at this phase in the acquisition cycle could introduce manufacturing risks. Stable decisions also assist in forecasting, impact analysis, component analysis, and other DMSMS-related activities. \n• The MRP system should have been uploaded, utilized and proven-out with the Pilot Line build. If the underlying information is even slightly off, then you will have problems with your MRP system. Past issues have been classified by the saying \"garbage in, garbage out.\" \n• The quality of the following information sources needs to be correct to prevent errors in your MRP system: (1) Master Production System (MPS); (2) Bills of Material (BOM); (3) Inventory and Inventory Status; (4) Lead Times; (5) Production Size; (6) Production Schedule (working times); (7) Quality Data (yield data, scrap, rework and repair, etc.); (8) Safety Stock Levels and Times; and (9) Work-in-Progress Data. \n• It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete. If no other options are available, a life-time buy of those parts/materials should be considered."]
[496,7,"H - Facilities","H.2 - Facilities","Have plans for the identified manufacturing facilities been developed to produce the Low Rate Initial Production (LRIP) build?","Purpose: \nTo ensure that materials planning has been properly developed to allow for duplicating a product configuration in a cost-effective manner, and that any risks to achieving this goal have been identified and addressed. MRLs play a vital role in ensuring that manufacturing processes have been effectively demonstrated in a pilot line environment and that industrial capabilities are reasonably available. In order to accomplish this, the material selection must be complete and stable, and an effective system to manage these materials needs to be established.","Additional Considerations: \n• What parts do we need to make or buy (Purchasing Plan)? \n• How many of these parts do we need (Capacity Plan)? \n• When must these parts be available (Detailed Manufacturing Schedule)? \n• Have you continued to check GIDEP alerts for obsolete or DMSMS materials?\n• The more complex the manufacturing operation the more complex the MRP. MRP systems generate two basic outputs, the Purchasing Plan and Schedule that lays out when the purchase orders (POs) should be released and when the purchased items should be received in order to support the production dates. \n• The second output is the Capacity Plan or Production Schedule. The production schedule details the start and completion dates for steps in the production process (routing) to include how many items will be produced in each batch and what is required from the bill of materials to support the fabrication and assembly. \n• The MRP system should be in place for the Pilot Line build.","Lessons Learned:\n• The Make/Buy decisions and BOM should be complete. Changes to these at this phase in the acquisition cycle could introduce manufacturing risks. Stable decisions also assist in forecasting, impact analysis, component analysis, and other DMSMS-related activities. \n• The MRP system should have been uploaded, utilized and proven-out with the Pilot Line build. If the underlying information is even slightly off, then you will have problems with your MRP system. Past issues have been classified by the saying \"garbage in, garbage out.\" \n• The quality of the following information sources needs to be correct to prevent errors in your MRP system: (1) Master Production System (MPS); (2) Bills of Material (BOM); (3) Inventory and Inventory Status; (4) Lead Times; (5) Production Size; (6) Production Schedule (working times); (7) Quality Data (yield data, scrap, rework and repair, etc.); (8) Safety Stock Levels and Times; and (9) Work-in-Progress Data. \n• It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete. If no other options are available, a life-time buy of those parts/materials should be considered."]
[497,7,"H - Facilities","H.2 - Facilities","Have requirements for human factors and ergonomics for manufacturing personnel, processes, and equipment been validated in a production representative environment?"]
[498,7,"H - Facilities","H.2 - Facilities","Have safety requirements for manufacturing personnel, processes, and equipment been validated in a production representative environment?","Purpose: \nTo ensure that an adequate manufacturing approach has been demonstrated in a low rate production environment and that adequate planning is available for cost-effective production at high rates.","\nAdditional Considerations: \n• What parts do we need to make or buy (Purchasing Plan)? \n• How many of these parts do we need (Capacity Plan)? \n• When must these parts be available (Detailed Manufacturing Schedule)?\n• Have you continued to check GIDEP alerts for obsolete or DMSMS materials?\n• The more complex the manufacturing operation the more complex the MRP. MRP systems generate two basic outputs, the Purchasing Plan and Schedule that lays out when the purchase orders (POs) should be released and when the purchased items should be received in order to support the production dates. \n• The second output is the Capacity Plan or Production Schedule. The production schedule details the start and completion dates for steps in the production process (routing) to include how many items will be produced in each batch and what is required from the bill of materials to support the fabrication and assembly.","Lessons Learned:\n• Increasing the production rate capability to FRP requirements effects many aspects of the manufacturing operations. Proactive planning is critical to ensure that efficient and economical production at these higher rates can be achieved. \n• The Make/Buy decisions and BOM must be stable for effective transition into a high rate production environment. Changes to these at this phase in the acquisition cycle could introduce manufacturing risks. Stable decisions also assist in forecasting, impact analysis, component analysis, and other DMSMS-related activities. \n• The MRP system should have been uploaded, utilized and proven-out with the LRIP build. If the underlying information is even slightly off, then you will have problems with your MRP system. Past issues have been classified by the saying \"garbage in, garbage out.\" \n• The quality of the following information sources needs to be correct to prevent errors in your MRP system: (1) Master Production System (MPS); (2) Bills of Material (BOM); (3) Inventory and Inventory Status; (4) Lead Times; (5) Production Size; (6) Production Schedule (working times); (7) Quality Data (yield data, scrap, rework and repair, etc.); (8) Safety Stock Levels and Times; and (9) Work-in-Progress Data. \n• It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete. If no other options are available, a life-time buy of those parts/materials should be considered."]
[499,8,"H - Facilities","H.2 - Facilities","Have pilot line facilities been successfully demonstrated?","Purpose: \nTo ensure that an adequate manufacturing approach has been demonstrated in a low rate production environment and that adequate planning is available for cost-effective production at high rates.","\nAdditional Considerations: \n• What parts do we need to make or buy (Purchasing Plan)? \n• How many of these parts do we need (Capacity Plan)? \n• When must these parts be available (Detailed Manufacturing Schedule)?\n• Have you continued to check GIDEP alerts for obsolete or DMSMS materials?\n• The more complex the manufacturing operation the more complex the MRP. MRP systems generate two basic outputs, the Purchasing Plan and Schedule that lays out when the purchase orders (POs) should be released and when the purchased items should be received in order to support the production dates. \n• The second output is the Capacity Plan or Production Schedule. The production schedule details the start and completion dates for steps in the production process (routing) to include how many items will be produced in each batch and what is required from the bill of materials to support the fabrication and assembly.","Lessons Learned:\n• Increasing the production rate capability to FRP requirements effects many aspects of the manufacturing operations. Proactive planning is critical to ensure that efficient and economical production at these higher rates can be achieved. \n• The Make/Buy decisions and BOM must be stable for effective transition into a high rate production environment. Changes to these at this phase in the acquisition cycle could introduce manufacturing risks. Stable decisions also assist in forecasting, impact analysis, component analysis, and other DMSMS-related activities. \n• The MRP system should have been uploaded, utilized and proven-out with the LRIP build. If the underlying information is even slightly off, then you will have problems with your MRP system. Past issues have been classified by the saying \"garbage in, garbage out.\" \n• The quality of the following information sources needs to be correct to prevent errors in your MRP system: (1) Master Production System (MPS); (2) Bills of Material (BOM); (3) Inventory and Inventory Status; (4) Lead Times; (5) Production Size; (6) Production Schedule (working times); (7) Quality Data (yield data, scrap, rework and repair, etc.); (8) Safety Stock Levels and Times; and (9) Work-in-Progress Data. \n• It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete. If no other options are available, a life-time buy of those parts/materials should be considered."]
[500,8,"H - Facilities","H.2 - Facilities","Are manufacturing facilities adequate and ready to begin Low Rate Initial Production (LRIP"]
[501,8,"H - Facilities","H.2 - Facilities","Are plans in place to support manufacturing facilities transition to Full Rate Production (FRP)?"]
[502,8,"H - Facilities","H.2 - Facilities","Is workplace safety meeting requirements (i.e., adequate)?","Purpose: \nTo ensure that the material selections are complete and stable, and that an adequate materials planning system is in place and proven to allow for continued cost-effective production at high rates.","Additional Considerations:\n• Have you continued to check GIDEP alerts for obsolete or DMSMS materials?","Lessons Learned:\n• It is important for engineers and program managers to be aware of GIDEP alerts that will inform them of parts and materials that are either obsolete or about to become obsolete. If no other options are available, a life-time buy of those parts/materials should be considered."]
[503,8,"H - Facilities","H.2 - Facilities","Have human factors and ergonomics for manufacturing personnel, processes, and equipment been demonstrated on a pilot line?"]
[504,8,"H - Facilities","H.2 - Facilities","Have safety requirements for manufacturing personnel, processes, and equipment been demonstrated on a pilot line?"]
[505,9,"H - Facilities","H.2 - Facilities","Have manufacturing facilities been successfully demonstrated in Low Rate Initial Production (LRIP)? "]
[506,9,"H - Facilities","H.2 - Facilities","Are manufacturing facility capacity plans adequate to meet the requirements for Full Rate Production (FRP)? "]
[507,9,"H - Facilities","H.2 - Facilities","Have human factors and ergonomics for manufacturing personnel, processes, and equipment been demonstrated during LRIP?"]
[508,9,"H - Facilities","H.2 - Facilities","Have safety requirements for manufacturing personnel, processes, and equipment been demonstrated during LRIP?"]
[509,10,"H - Facilities","H.2 - Facilities","Are the production facilities in place to meet Full Rate Production (FRP) requirements?"]
[510,10,"H - Facilities","H.2 - Facilities","Has the capacity of the production facilities been demonstrated to meet maximum Full Rate Production (FRP) requirements?"]
[511,10,"H - Facilities","H.2 - Facilities","Have human factors and ergonomics for manufacturing personnel, processes, and equipment been demonstrated during FRP?"]
[512,10,"H - Facilities","H.2 - Facilities","Have safety requirements for manufacturing personnel, processes, and equipment been demonstrated during FRP?"]
[513,1,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Have manufacturing management considerations been surveyed and included in early planning activities?"]
[514,2,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Have manufacturing management needs been assessed, analyzed and validated?"]
[515,3,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Have manufacturing management requirements for system concepts been identified?"]
[516,4,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Has the manufacturing strategy for the preferred materiel solution been developed?"]
[517,4,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Has the manufacturing strategy for the preferred materiel solution been considered in the AoA?"]
[518,4,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Has the manufacturing strategy for the preferred materiel solution been incorporated into the Acquisition Strategy?"]
[519,4,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Have the prototype schedule risk mitigation efforts been documented in the SEP?"]
[520,5,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Based upon the preferred concept, has the manufacturing strategy been refined?"]
[521,5,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Have mitigation efforts concerning schedule risk of prototypes (i.e., testing and delivery schedules) been initiated?"]
[522,6,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Has the initial manufacturing approach for the next phase (EMD) been developed?"]
[523,6,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Are all system design related manufacturing events included in the Integrated Master Plan/Schedule (IMP/IMS)?"]
[524,6,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","For either the pilot line and/or technology insertion, has a mitigation approach for manufacturing risks and issues been defined?"]
[525,7,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Has the initial manufacturing plan been developed?"]
[526,7,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Have manufacturing events from the initial Manufacturing Plan been included in the Integrated Master Plan and/or the Integrated Master Schedule?"]
[527,7,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Have manufacturing risks and issues been integrated into program mitigation plans?"]
[528,7,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Have initial work instructions been developed? "]
[529,7,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Is an effective production control system in place to support the pilot line? "]
[530,8,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Has the manufacturing plan been updated for Low Rate Initial Production (LRIP)?"]
[531,8,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Have all manufacturing risks and issues been identified and assessed?"]
[532,8,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Do all identified manufacturing risks and issues have approved mitigation plans in place?"]
[533,8,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Have work instructions been finalized? "]
[534,8,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Is an effective production control system in place to support Low Rate Initial Production (LRIP)?"]
[535,9,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Has the manufacturing plan been updated for Full Rate Production (FRP)?"]
[536,9,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Are all manufacturing risks and issues being managed?"]
[537,9,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Is an effective production control system in place to support Full Rate Production (FRP)?"]
[538,10,"I - Manufacturing Management","I.1 - Manufacturing Planning & Scheduling","Are all manufacturing risks and issues being managed?"]
[539,1,"I - Manufacturing Management","I.2 - Materials Planning","Has materials planning state of the art been surveyed?"]
[540,2,"I - Manufacturing Management","I.2 - Materials Planning","Have initial availability, lead time, handling and storage requirements for potential materials and components been evaluated?"]
[541,3,"I - Manufacturing Management","I.2 - Materials Planning","Has a materials and components list for system concepts been developed?"]
[542,3,"I - Manufacturing Management","I.2 - Materials Planning","Have initial materials planning requirements (i.e., availability, lead times, handling, and storage) been identified?"]
[543,4,"I - Manufacturing Management","I.2 - Materials Planning","Has the materials and components list with estimates for availability, lead times, handling and storage requirements been developed and considered in the AoA?"]
[544,5,"I - Manufacturing Management","I.2 - Materials Planning","Have make or buy evaluations been initiated?"]
[545,5,"I - Manufacturing Management","I.2 - Materials Planning","Do the make or buy evaluations include requirements and considerations for pilot line, initial production (LRIP), and rate production (FRP)? "]
[546,5,"I - Manufacturing Management","I.2 - Materials Planning","Have lead times and other materials risks and issues been identified?"]
[547,6,"I - Manufacturing Management","I.2 - Materials Planning","Have most make or buy trade studies been completed?"]
[548,6,"I - Manufacturing Management","I.2 - Materials Planning","Have materials risks and issues been assessed (identified)? "]
[549,6,"I - Manufacturing Management","I.2 - Materials Planning","Have mitigation plans for the identified materials risks and issues been developed?"]
[550,6,"I - Manufacturing Management","I.2 - Materials Planning","Has the Bill of Materials (BOM) been initiated?"]
[551,7,"I - Manufacturing Management","I.2 - Materials Planning","Have all the make or buy decisions been completed for pilot line build?"]
[552,7,"I - Manufacturing Management","I.2 - Materials Planning","Is the Bill of Materials completed for pilot line build? "]
[553,7,"I - Manufacturing Management","I.2 - Materials Planning","Are material planning systems in place for the pilot line build?"]
[554,8,"I - Manufacturing Management","I.2 - Materials Planning","Based on the pilot line, are there any changes to the make or buy decisions for initial production (LRIP)?  "]
[555,8,"I - Manufacturing Management","I.2 - Materials Planning","Based on pilot line, are there any changes to the Bill of Materials for initial production (LRIP)?  "]
[556,8,"I - Manufacturing Management","I.2 - Materials Planning","Have material planning systems (i.e., MRP II, ERP, etc.) been proven on the pilot line and are ready for Low Rate Initial Production (LRIP) build?"]
[557,9,"I - Manufacturing Management","I.2 - Materials Planning","Based on initial production, are there any changes to the make or buy decisions for Full Rate Production (FRP)?"]
[558,9,"I - Manufacturing Management","I.2 - Materials Planning","Based on initial production, are there any changes to the Bill of Materials for Full Rate Production (FRP)?"]
[559,9,"I - Manufacturing Management","I.2 - Materials Planning","Have material planning systems (i.e., MRP II, ERP, etc.) been proven in LRIP?"]
[560,9,"I - Manufacturing Management","I.2 - Materials Planning","Is the material planning systems (i.e., MRP II, ERP, etc.) sufficient for Full Rate Production (FRP)?"]
[561,10,"I - Manufacturing Management","I.2 - Materials Planning","Have the material planning systems (i.e., MRP II, ERP, etc.) been validated by FRP?"]

{"sheet":"Summary Pivot","index":10,"n_rows":442,"n_cols":4,"rows_kept":442,"cut":false,"hidden":true,"merged":[]}
[]
["Current Revision of MRL Matrix Revision","v.2016"]
["Current User's Guide Revision","v.2016"]
[]
["MRL FILTER *(Select the MRL Levels you would like to display by clicking the down arrow in the cell B4) ","(All)"]
[]
["Thread \n*(Select the threads you would like to display by clicking the down arrow in this cell A6)","MRL","Question","Meets Criteria"]
["A - Technology & Industrial Base"]
["A.0 - Technology Maturity",1,"Is the Technology Readiness at TRL 1 or greater?","(blank)"]
[null,2,"Is the Technology Readiness at TRL 2 or greater?","(blank)"]
[null,3,"Is the Technology Readiness at TRL 3 or greater?","(blank)"]
[null,4,"Is the Technology Readiness at TRL 4 or greater?","(blank)"]
[null,5,"Is the Technology Readiness at TRL 5 or greater?","(blank)"]
[null,6,"Is the Technology Readiness at TRL 6 or greater?","(blank)"]
[null,7,"Is the Technology Readiness at TRL 7 or 8?","(blank)"]
[null,8,"Is the Technology Readiness at TRL 8 or 9?","(blank)"]
[null,9,"Is the Technology Readiness at TRL 9?","(blank)"]
[null,10,"Is the Technology Readiness at TRL 9?","(blank)"]
["A.1 - Technology Transition to Production",3,"Have potential manufacturing sources been identified for technology needs (Understand state of the art)?","(blank)"]
[null,4,"Have industrial base capabilities and gaps/risks been identified for key technologies, components, and/or key processes?","(blank)"]
[null,5,"Has planning begun to minimize the risks associated with sole/single/foreign source vendors?","(blank)"]
[null,null,"Has the industrial base capabilities assessment been initiated to identify potential manufacturing sources to produce the required capability?","(blank)"]
[null,null,"Have sole/single/foreign source vendors and vendors of technologies with potential obsolescence issues been identified? ","(blank)"]
[null,6,"Are the plans to minimize sole/single/foreign sources and obsolescence issues complete?","(blank)"]
[null,null,"Has an industrial base capabilities assessment for MS B been completed?","(blank)"]
[null,null,"Has the need for sole/single/foreign sources been justified?","(blank)"]
[null,null,"Have potential alternative sources been identified?","(blank)"]
[null,null,"Is the industrial capability in place to support the manufacturing of development articles?","(blank)"]
[null,7,"Are the necessary alternates to sole/single/foreign sources being developed?","(blank)"]
[null,null,"Has the industrial capability to support production been analyzed?","(blank)"]
[null,null,"Is the stability of all sole/single/foreign sources and obsolescence issues being assessed/monitored?","(blank)"]
[null,8,"Are all necessary sources available including multi-sourcing where cost-effective or necessary to mitigate risk?","(blank)"]
[null,null,"Has the industrial base capability assessment for Milestone C been completed?","(blank)"]
[null,null,"Is the industrial capability in place to support Low Rate Initial Production (LRIP)?","(blank)"]
[null,9,"Is the industrial capability assessment for FRP complete?","(blank)"]
[null,null,"Is the Industrial capability in place to support the start of FRP?","(blank)"]
[null,10,"Does the Industrial Capability support FRP","(blank)"]
[null,null,"Has the industrial capability been assessed to support modifications, upgrades, surge, and other potential manufacturing requirements?","(blank)"]
["A.2 - Manufacturing Technology Development",2,"Have new manufacturing concepts and potential solutions been identified?","(blank)"]
[null,3,"Have manufacturing technology concepts been identified through experiments/models?","(blank)"]
[null,4,"Have pertinent Manufacturing Science (MS) and Advanced Manufacturing Technology requirements been identified?","(blank)"]
[null,5,"Have the required manufacturing technology development efforts been initiated as applicable?","(blank)"]
[null,6,"Are the necessary manufacturing technology development efforts continuing?","(blank)"]
[null,null,"Have the required manufacturing technology development solutions been demonstrated in a production relevant environment?","(blank)"]
[null,7,"Are the necessary manufacturing technology development efforts still continuing?","(blank)"]
[null,null,"Have the required manufacturing science and manufacturing technology development solutions been demonstrated in a production representative environment?","(blank)"]
[null,8,"Are the primary manufacturing technology efforts concluding and some improvement efforts continuing?","(blank)"]
[null,null,"Have the required manufacturing technology development solutions been validated on a pilot line?","(blank)"]
[null,9,"Have the required manufacturing technology process improvement efforts been initiated for Full Rate Production (FRP)?","(blank)"]
[null,10,"Are manufacturing technology continuous process improvements ongoing?","(blank)"]
["B - Design"]
["B.1 - Producibility Program",3,"Have relevant materials/processes been evaluated for manufacturability using experiments/models?","(blank)"]
[null,4,"Are initial producibility and manufacturability assessments of preferred systems concepts completed?","(blank)"]
[null,null,"Are the results of the producibility and manufacturability assessment being considered in the selection of preferred design concepts?","(blank)"]
[null,null,"Are the results of the producibility and manufacturability assessment being reflected in the Technology Development Strategy key components/technologies?","(blank)"]
[null,5,"Do ongoing design trades consider manufacturing processes and industrial base capability constraints? ","(blank)"]
[null,null,"Have producibility & manufacturability assessments of key technologies and components been initiated as appropriate?","(blank)"]
[null,null,"Have the manufacturing processes been assessed for capability to test and verify in production and their influence on Operations and Support (O&S)?","(blank)"]
[null,6,"Are the preliminary design choices assessed against manufacturing processes and industrial base capability constraints? ","(blank)"]
[null,null,"Are the producibility enhancement efforts (e.g., Design For Manufacturing and Design for Assembly) initiated?","(blank)"]
[null,null,"Have producibility assessments and producibility trade studies (performance vs. producibility) of key technologies/components been completed?","(blank)"]
[null,null,"Are the results of the producibility assessments and trade studies being used to shape the Acquisition Strategy, Systems Engineering Plan (SEP), Manufacturing and Producibility plans, and Planning for Engineering and Manufacturing Development (EMD) or tec","(blank)"]
[null,7,"Are manufacturing processes re-assessed as needed for capability to test and verify potential influence on Operations & Support? ","(blank)"]
[null,null,"Are the producibility enhancement efforts (e.g. Design for Mfg., Assembly, etc. - DFX) ongoing for an optimized integrated system? ","(blank)"]
[null,8,"Have the known producibility issues been resolved and pose no significant risk for Low Rate Initial Production (LRIP)?","(blank)"]
[null,null,"Have the producibility improvements been implemented on the system?","(blank)"]
[null,9,"Have prior producibility improvements been analyzed for effectiveness during Low Rate Initial Production (LRIP)?","(blank)"]
[null,null,"Have the producibility issues/risks discovered in Low Rate Initial Production (LRIP) been mitigated and pose no significant risk for Full Rate Production (FRP)?","(blank)"]
[null,10,"Are all modifications, upgrades, Diminishing Mfg Sources & Material Shortages (DMSMS), and other changes assessed for producibility?","(blank)"]
[null,null,"Are process producibility improvements ongoing?","(blank)"]
[null,null,"Have design producibility improvements been demonstrated in Full Rate Production (FRP)?","(blank)"]
["B.2 - Design Maturity",1,"Have manufacturing research opportunities been identified?","(blank)"]
[null,2,"Have applications been defined?","(blank)"]
[null,null,"Have broad performance goals been identified that may drive manufacturing options?","(blank)"]
[null,3,"Are product lifecycle requirements and technical requirements being evaluated?","(blank)"]
[null,null,"Have top level performance requirements been defined?","(blank)"]
[null,null,"Have trade-offs in design options been assessed based on experiments?","(blank)"]
[null,4,"Are form, fit, and function constraints and manufacturing capabilities identified for the preferred systems concepts?","(blank)"]
[null,null,"Are system characteristics and measures to support required capabilities identified?","(blank)"]
[null,null,"Do the Systems Engineering Plan (SEP) and the Test and Evaluation Strategy recognize the need for the establishment/validation of manufacturing capability and management of manufacturing risk for the product lifecycle?","(blank)"]
[null,null,"Have initial potential Key Performance Parameters (KPPs) been identified for the preferred systems concept?","(blank)"]
[null,5,"Are all enabling/critical technologies and components identified? ","(blank)"]
[null,null,"Are lower level performance requirements sufficient to proceed to the preliminary design?  ","(blank)"]
[null,null,"Do enabling/critical technologies and components consider the product lifecycle? ","(blank)"]
[null,null,"Have product data required for prototype component manufacturing been released?","(blank)"]
[null,null,"Have the evaluation of design Key Characteristics (KC) been initiated? ","(blank)"]
[null,6,"Are the product requirements and features well enough defined to support Preliminary Design Review?","(blank)"]
[null,null,"Has a system allocated baseline been established?","(blank)"]
[null,null,"Has product data essential for  subsystem/system prototyping been released?","(blank)"]
[null,null,"Have all enabling/critical technologies/components been demonstrated?","(blank)"]
[null,null,"Have preliminary design Key Characteristics (KC) been defined and are mitigation plans in development?","(blank)"]
[null,7,"Are the product design and features well enough defined to support critical design review even though design change traffic may be significant?","(blank)"]
[null,null,"Has all the product data essential for component manufacturing been released?","(blank)"]
[null,null,"Have potential Key Characteristic risk issues been identified?","(blank)"]
[null,null,"Have preliminary design Key Characteristic (KC) issues been defined and are mitigation plans in development?","(blank)"]
[null,8,"Are the Key Characteristics attainable based upon pilot line demonstrations?","(blank)"]
[null,null,"Does the design change traffic have minimal impact on Low Rate Initial Production (LRIP)?","(blank)"]
[null,null,"Has all the product data essential for system manufacturing been released?","(blank)"]
[null,null,"Is the detailed design of product features and interfaces complete?","(blank)"]
[null,9,"Are all Key Characteristics controlled in Low Rate Initial Production (LRIP) to appropriate quality levels?","(blank)"]
[null,null,"Are the major product design features and configuration stable?","(blank)"]
[null,null,"Has the system design been validated through operational testing of Low Rate Initial Production (LRIP) items?","(blank)"]
[null,null,"Is the design change traffic limited to minor configuration changes?","(blank)"]
[null,null,"Is the Physical Configuration Audit (PCA) or equivalent complete as necessary?","(blank)"]
[null,10,"Are all Key Characteristics controlled in Full Rate Production (FRP) to appropriate quality levels?","(blank)"]
[null,null,"Is the product design stable (i.e. design changes are few and generally limited to those required for continuous improvement or in reaction to obsolescence)? ","(blank)"]
["C - Cost & Funding"]
["C.1 - Production Cost Knowledge (Cost modeling)",2,"Has the cost model approach been defined?","(blank)"]
[null,3,"Has a high level process chart model been developed? ","(blank)"]
[null,null,"Have initial cost targets and risks been identified? ","(blank)"]
[null,null,"Have technology cost models been developed for new process steps and materials based on experiments?","(blank)"]
[null,4,"Are detailed process chart cost models driven by process variables?","(blank)"]
[null,null,"Has cost driver uncertainty been quantified?","(blank)"]
[null,null,"Have key manufacturing, material and specialized requirement cost drivers been identified?    ","(blank)"]
[null,5,"Do prototype components or simulations produced in a production relevant environment drive the formulation of the final production cost model?","(blank)"]
[null,null,"Is there a realistic cost model that includes  materials, labor, equipment, tooling/Special Test Equipment (STE), setup, yield/scrap/rework, Work In Progress (WIP), and capability/capacity constraints?","(blank)"]
[null,6,"Have cost models been updated with design requirements, material specifications, tolerances, integrated master schedule, results of system/subsystem simulations and  production relevant prototype demonstrations?  ","(blank)"]
[null,7,"Are cost models updated with the results of systems/sub-systems produced in a production representative environment, production plant layout and design, and obsolescence solutions?","(blank)"]
[null,8,"Have cost models been updated with results of the pilot line build?","(blank)"]
[null,9,"Has the Full Rate Production (FRP) cost model been updated with the result of the Low Rate Initial Production (LRIP) build?","(blank)"]
[null,10,"Has the cost model been validated against actual Full Rate Production (FRP) cost?","(blank)"]
["C.2 - Cost Analysis",1,"Have manufacturing cost implications been identified?","(blank)"]
[null,2,"Have cost elements been identified?","(blank)"]
[null,3,"Has a sensitivity analysis been conducted to define cost drivers and production development strategy (i.e. lab to pilot to factory)?","(blank)"]
[null,4," Do initial cost models support Analysis of Alternatives (AoA) and Alternative Systems Review (ASR)?","(blank)"]
[null,null,"Have producibility cost risks been assessed?  ","(blank)"]
[null,5,"Are costs analyzed using prototype component actuals to ensure target costs are achievable? ","(blank)"]
[null,null,"Are decisions regarding design choices, make/buy, capacity, process capability, sources, quality, key characteristics, yield/rate, and variability influenced by cost models?   ","(blank)"]
[null,6,"Are costs analyzed using prototype system/sub-system actuals to ensure target costs are achievable? ","(blank)"]
[null,null,"Have cost reduction and avoidance strategies been developed?","(blank)"]
[null,null,"Have cost targets been allocated to subsystems?","(blank)"]
[null,null,"Have the manufacturing cost drivers for \"Should-Cost\" models been provided?","(blank)"]
[null,7," Are detailed trade studies and engineering change requests supported by cost estimates?","(blank)"]
[null,null,"Are cost reduction and avoidance strategies underway?","(blank)"]
[null,null,"Have the manufacturing cost drivers for \"Should-Cost\" models been updated?","(blank)"]
[null,null,"Have the manufacturing costs been rolled up to the system/sub-system level and been tracked against targets?","(blank)"]
[null,8,"Are costs analyzed using pilot line actuals to ensure target costs are achievable? ","(blank)"]
[null,null,"Are the cost reduction initiatives ongoing?","(blank)"]
[null,null,"Does manufacturing cost analysis support proposed changes to requirements or configuration?","(blank)"]
[null,9,"Are the cost reduction initiatives still ongoing?","(blank)"]
[null,null,"Has touch labor efficiency been analyzed to meet production rates?","(blank)"]
[null,null,"Have elements of inefficiency for touch labor been identified with plans in place for reduction?","(blank)"]
[null,null,"Have Low Rate Initial Production (LRIP) cost goals been met and has the learning curve been analyzed with actual data?  ","(blank)"]
[null,null,"Have the manufacturing cost drivers for \"Should-Cost\" models been updated?","(blank)"]
[null,10,"Are the cost reduction initiatives continuing?","(blank)"]
[null,null,"Have Full Rate Production (FRP) cost goals been met?","(blank)"]
["C.3 - Manufacturing Investment Budget",1,"Have potential investments been identified?","(blank)"]
[null,2,"Do the program/projects have reasonable budget estimates for reaching MRL 3 through experiment?","(blank)"]
[null,3,"Do the program/projects have reasonable budget estimates for reaching MRL 4 by Milestone A?","(blank)"]
[null,4,"Are all outstanding MRL 4 risk areas understood with approved mitigation plans in place?","(blank)"]
[null,null,"Does the cost estimate include capital investment for production-relevant equipment?","(blank)"]
[null,null,"Does the program have a reasonable budget estimate to reach MRL 6 by Milestone B?","(blank)"]
[null,null,"Have manufacturing technology initiatives been identified to reduce costs?","(blank)"]
[null,5,"Are all outstanding MRL 5 risk areas understood with approved mitigation plans in place?","(blank)"]
[null,null,"Does the program have an updated budget estimate for reaching MRL 6 by Milestone B?","(blank)"]
[null,6,"Are all outstanding MRL 6 risk areas understood with approved mitigation plans in place?","(blank)"]
[null,null,"Does the cost estimate include capital investment for production-representative equipment by CDR and pilot line equipment by Milestone C?","(blank)"]
[null,null,"Does the program have a reasonable budget estimate for reaching MRL 8 by Milestone C?","(blank)"]
[null,7,"Are all outstanding MRL 7 risk areas understood with approved mitigation plans in place?","(blank)"]
[null,null,"Does the program have an updated budget estimate for reaching MRL 8 by Milestone C?","(blank)"]
[null,8,"Are all outstanding MRL 8 risk areas understood with approved mitigation plans in place?","(blank)"]
[null,null,"Does the cost estimate include investment for Low Rate Initial Production (LRIP) and Full Rate Production (FRP)?","(blank)"]
[null,null,"Does the program have a reasonable budget estimate for reaching MRL 9 by the Full Rate Production (FRP) decision point?","(blank)"]
[null,9,"Are all outstanding MRL 9 risk areas understood with approved mitigation plans in place?","(blank)"]
[null,null,"Does the program have a reasonable budget estimate for Full Rate Production (FRP)?","(blank)"]
[null,10,"Are the production budgets sufficient for production at the required rates and schedule to support the funded program?","(blank)"]
["D - Materials"]
["D.1 - Maturity",1,"Have material properties been identified for research?","(blank)"]
[null,2,"Have material properties and characteristics been predicted?","(blank)"]
[null,3,"Have material properties been validated and assessed for basic manufacturability using experiments?","(blank)"]
[null,4,"Have projected materials been produced in a laboratory environment?","(blank)"]
[null,5,"Are maturation efforts in place to address new material production risks for technology demonstration?","(blank)"]
[null,null,"Have all materials been manufactured or produced in a prototype environment (may be in a similar application/program)?","(blank)"]
[null,6,"Are preliminary material specifications in place?","(blank)"]
[null,null,"Has the material maturity been verified with technology demonstration articles?","(blank)"]
[null,null,"Have material properties been adequately characterized?","(blank)"]
[null,7,"Have the necessary material specifications been approved?","(blank)"]
[null,null,"Is the material maturity sufficient for a pilot line build?","(blank)"]
[null,8,"Are material specifications stable?","(blank)"]
[null,null,"Have the materials been proven and validated during Engineering and Manufacturing Development (EMD) as adequate to support Low Rate Initial Production (LRIP)?","(blank)"]
[null,9,"Are the materials proven and validated as adequate to support FRP?","(blank)"]
[null,null,"Have all necessary materials been proven and controlled to specification in Low Rate Initial Production (LRIP)?","(blank)"]
[null,10,"Have all necessary materials been proven and controlled to specification in Full Rate Production (FRP)?","(blank)"]
["D.2 - Availability",2,"Has material availability been assessed?","(blank)"]
[null,3,"Have material scale-up issues been identified?","(blank)"]
[null,4,"Have material quantities and lead times been estimated?","(blank)"]
[null,null,"Have projected lead times been identified for all difficult to obtain, difficult to process, or hazardous materials?","(blank)"]
[null,5,"Has planning begun to address scale-up issues?","(blank)"]
[null,null,"Have material availability issues been addressed for the prototype build?","(blank)"]
[null,null,"Have significant material risks been identified for all materials?","(blank)"]
[null,6,"Have components been assessed for future DMSMS risk?","(blank)"]
[null,null,"Have Long-lead items been identified?","(blank)"]
[null,null,"Have material availability issues been addressed to meet Engineering and Manufacturing Development (EMD) build?","(blank)"]
[null,null,"Have potential obsolescence issues been identified?","(blank)"]
[null,7,"Are mitigation strategies in place for DMSMS components?","(blank)"]
[null,null,"Have long lead procurements been identified/planned for Low Rate Initial Production (LRIP)?","(blank)"]
[null,null,"Is an obsolescence plan in place?","(blank)"]
[null,8,"Are material availability risk issues minimized for Low Rate Initial Production (LRIP)?","(blank)"]
[null,null,"Have long lead procurements been initiated for Low Rate Initial Production (LRIP)?","(blank)"]
[null,null,"Have availability Issues been addressed to meet FRP builds?","(blank)"]
[null,9,"Are availability risk issues minimized for Full Rate Production (FRP)?","(blank)"]
[null,null,"Have long lead procurements been initiated for Full Rate Production (FRP)?","(blank)"]
[null,10,"Are all significant Full Rate Production (FRP) material availability risk issues resolved?","(blank)"]
["D.3 - Supply Chain Management",3,"Has an initial assessment of potential supply chain capability been completed?","(blank)"]
[null,4,"Has a survey for potential supply chain sources been completed?","(blank)"]
[null,5,"Has a survey for potential supply chain sources been completed?\n\n","(blank)"]
[null,null,"\nHave potential supply chain sources been identified and evaluated as able to support prototype build?\n","(blank)"]
[null,6,"Are supply chain plans (e.g., teaming agreements, etc.) in place leading to an Engineering and Manufacturing Development (EMD) contract award?","(blank)"]
[null,null,"Have life cycle supply chain requirements and the critical suppliers list been updated?","(blank)"]
[null,7,"Has a plan been developed for establishing predictive indicators?","(blank)"]
[null,null,"Is there an effective supply chain management process defined, documented and in place?","(blank)"]
[null,null,"Is there an effective process in place for management of suppliers using metrics for tracking and managing supplier performance?","(blank)"]
[null,8,"Has an adequate assessment of the critical second and lower tier supply chain been completed?","(blank)"]
[null,null,"Has supplier compliance with program requirements and changes been validated?","(blank)"]
[null,null,"Has the plan for predictive indicators been updated for use in production?","(blank)"]
[null,null,"Is the supply chain adequate to support Low Rate Initial Production (LRIP)?","(blank)"]
[null,null,"Is there a robust and disciplined requirements flow-down process verified and in place?","(blank)"]
[null,9,"Are long term agreements in place where practical?","(blank)"]
[null,null,"Are predictive indicators in place to manage suppliers?","(blank)"]
[null,null,"Are the Prime's supplier management metrics (including thresholds and goals) in place and being used to manage risk?","(blank)"]
[null,null,"Is the supply chain stable and adequate to support Full Rate Production (FRP)?","(blank)"]
[null,10,"Is the supply chain stable and adequate to support Full Rate Production (FRP)?","(blank)"]
["D.4 - Special Handling",2,"Has an initial evaluation of potential regulatory requirements and special handling concerns been completed?","(blank)"]
[null,3,"Has an initial evaluation of potential regulatory requirements and special handling concerns been completed?","(blank)"]
[null,null,"Have potential special handling procedures been applied in the lab?","(blank)"]
[null,null,"Have special handling concerns been assessed?","(blank)"]
[null,4,"Has a list of hazardous materials been updated?","(blank)"]
[null,null,"Have special handling concerns been assessed?","(blank)"]
[null,null,"Have special handling requirements been identified?","(blank)"]
[null,null,"Have the special handling procedures been applied in the lab?","(blank)"]
[null,5,"Have all special handling requirement gaps been identified?","(blank)"]
[null,null,"Have new special handling processes been demonstrated in lab environment?","(blank)"]
[null,null,"Have special handling procedures been applied in a production relevant environment?","(blank)"]
[null,6,"Have special handling procedures been applied in a production relevant environment?","(blank)"]
[null,null,"Have the plans to address the special handling requirement gaps been completed?","(blank)"]
[null,7,"Have special handling procedures been applied in a production representative environment?","(blank)"]
[null,null,"Have special handling procedures been developed and annotated on work instructions for the pilot line?","(blank)"]
[null,8,"Do all work instructions contain special handling provisions as required?","(blank)"]
[null,null,"Have special handling procedures been applied in a pilot line environment?","(blank)"]
[null,null,"Have special handling procedures been demonstrated in Engineering and Manufacturing Development (EMD) or technology insertion programs?","(blank)"]
[null,null,"Have the special handling risk issues been minimized for Low Rate Initial Production (LRIP)?","(blank)"]
[null,9,"Have special handling procedures been applied in a Low Rate Initial Production (LRIP) environment?","(blank)"]
[null,null,"Have the special handling procedures been demonstrated in Low Rate Initial Production (LRIP)?","(blank)"]
[null,null,"Have the special handling risk issues been minimized for Full Rate Production (FRP)?","(blank)"]
[null,10,"Have the special handling procedures been effectively implemented in Full Rate Production (FRP)?  ","(blank)"]
["E - Process Capability & Control"]
["E.1 - Modeling & Simulation",2,"Have initial models been developed, if applicable?","(blank)"]
[null,3,"Have proposed manufacturing concepts or producibility needs been identified based on high-level process flow chart models?","(blank)"]
[null,4,"Have production modeling and simulation approaches for process or product been identified?","(blank)"]
[null,5,"Have the initial simulation models (product or process) been developed at the component level and used to determine constraints?","(blank)"]
[null,6,"Have the initial simulation models been developed at the technology sub-system or system level and used to determine system constraints?","(blank)"]
[null,7,"Have the simulation models been used to determine system constraints and identify improvement opportunities?","(blank)"]
[null,8,"Have the results of the simulation models been used to improve processes and determine that Low Rate Initial Production (LRIP) requirements can be met?","(blank)"]
[null,null,"Have the simulation models been verified by the pilot line? ","(blank)"]
[null,9,"Have the results of the simulation models been used to determine if Full Rate Production (FRP) requirements can be met?","(blank)"]
[null,null,"Have the simulation models been verified by Low Rate Initial Production (LRIP) build and used to assist in management of Low Rate Initial Production (LRIP)?","(blank)"]
[null,10,"Have the production simulation models been used as a tool to assist in management of Full Rate Production (FRP)?","(blank)"]
[null,null,"Have the simulation models been verified by Full Rate Production (FRP) build?","(blank)"]
["E.2 - Manufacturing Process Maturity",2,"Have material and/or process approaches been identified?","(blank)"]
[null,3,"Have critical manufacturing processes been identified through experimentation?","(blank)"]
[null,null,"Have high level manufacturing processes been documented?","(blank)"]
[null,4,"Has a survey been completed to determine the current state of critical processes?","(blank)"]
[null,5,"Has process maturity been assessed on similar processes in production?","(blank)"]
[null,null,"Have process capability requirements been identified for pilot line, Low Rate Initial Production (LRIP) and Full Rate Production (FRP)?","(blank)"]
[null,6,"Has collecting or estimating process capability data from prototype build begun, and have process capability requirements been refined? ","(blank)"]
[null,null,"Have the key manufacturing processes been demonstrated in a production relevant environment?","(blank)"]
[null,7,"Has collecting or estimating process capability data continued? ","(blank)"]
[null,null,"Have process capability requirements been refined?","(blank)"]
[null,null,"Have the manufacturing processes been successfully demonstrated in a production representative environment?","(blank)"]
[null,8,"Does capability data from the pilot line meet the target?","(blank)"]
[null,null,"Have process capability requirements for LRIP been refined based on Pilot Line data?","(blank)"]
[null,null,"Have the manufacturing processes for Low Rate Initial Production (LRIP) been verified on a pilot line?","(blank)"]
[null,9,"Are the manufacturing processes stable, adequately controlled, capable, and have the manufacturing processes achieved program  Low Rate Initial Production (LRIP) objectives?","(blank)"]
[null,null,"Have variability experiments been conducted to show Full Rate Production (FRP) impact and potential for continuous improvement? ","(blank)"]
[null,10,"Are the manufacturing processes stable, adequately controlled, capable, and have the manufacturing processes achieved program  Full Rate Production (FRP) objectives?","(blank)"]
["E.3 - Process Yields and Rates",3,"Have initial estimates of yields and rates based on experiments or state of the art been completed?","(blank)"]
[null,4,"Has a yield and rates assessment on proposed/similar processes been applied within the Analysis of Alternatives (AoA)?","(blank)"]
[null,null,"Has a yield and rates assessment on proposed/similar processes been completed?","(blank)"]
[null,5,"Have target yield and rates been established for the pilot line, Low Rate Initial Production (LRIP), and Full Rate Production (FRP)?","(blank)"]
[null,null,"Have yield and rate improvement plans been developed/initiated?","(blank)"]
[null,null,"Have yield and rate issues been identified and does the detailed production schedule support the required production rate?","(blank)"]
[null,6,"Are results from the yields and rates from the production relevant environment being fed into an improvement plan?","(blank)"]
[null,null,"Are yields and rates from the production relevant environment being evaluated against targets?","(blank)"]
[null,7,"Are results from the yields and rates from the production representative environment being fed into  improvement plans?","(blank)"]
[null,null,"Are yields and rates from the production representative environment being evaluated against pilot line targets?","(blank)"]
[null,8,"Are yield and rate improvement plans ongoing and being updated?","(blank)"]
[null,null,"Have pilot line yield and rate targets been achieved?","(blank)"]
[null,null,"Have yields and rates required to begin Low Rate Initial Production (LRIP) been refined using pilot line articles?","(blank)"]
[null,9,"Are yield improvements on-going?","(blank)"]
[null,null,"Have the Low Rate Initial Production (LRIP) yield and rate targets been achieved?","(blank)"]
[null,null,"Have the results of LRIP yields and rates been used to refine FRP yield requirements?","(blank)"]
[null,10,"Are yield improvements on-going?","(blank)"]
[null,null,"Have the Full Rate Production (FRP) yield and rate targets been achieved?","(blank)"]
["F - Quality Management"]
["F.1 - Quality Management including Supplier Quality",4,"Has a quality strategy been identified as part of the Acquisition Strategy (AS)?","(blank)"]
[null,null,"Is the updated quality strategy included in the System Engineering Plan (SEP)?","(blank)"]
[null,5,"Has the quality strategy been updated to reflect Key Characteristic identification activities?","(blank)"]
[null,6,"Have quality risks and adequate metrics been identified and improvement plans initiated?","(blank)"]
[null,null,"Is there an initial Quality Plan (QP) and a Quality Management System (QMS) in place?","(blank)"]
[null,7,"Do Quality Management System (QMS) elements (e.g., control of nonconforming material, corrective action, etc.) meet requirements of appropriate industry standards?","(blank)"]
[null,null,"Have the quality targets been established?","(blank)"]
[null,null,"Is a program-specific quality plan being developed?","(blank)"]
[null,8,"Have quality targets been assessed against the pilot line to see if they drive continuous quality improvement?","(blank)"]
[null,null,"Have the program-specific quality plan and Quality Manager been established?","(blank)"]
[null,9,"Have quality targets been verified on the Low Rate Initial Production (LRIP) line?","(blank)"]
[null,null,"Is continuous quality improvement on-going?","(blank)"]
[null,null,"Is management review of quality targets conducted on a regular basis with appropriate action being taken?","(blank)"]
[null,10,"Are statistical controls being applied where appropriate?","(blank)"]
[null,null,"Have the quality targets been verified on the Full Rate Production (FRP) line?","(blank)"]
[null,null,"Is continuous quality improvement on-going?","(blank)"]
["F.2 - Product Quality",4,"Has a product inspection and acceptance testing strategy been identified as part of the Acquisition Strategy?","(blank)"]
[null,null,"Has a product inspection and acceptance testing strategy been included in the Systems Engineering Plan (SEP)?","(blank)"]
[null,5,"Have roles and responsibilities been identified for acceptance test procedures, in-process and final inspections? ","(blank)"]
[null,null,"Have statistical process controls been identified for prototype units?","(blank)"]
[null,null,"Is there a program in place to identify and manage key characteristics?  ","(blank)"]
[null,6,"Has a Key Characteristic management approach been defined?","(blank)"]
[null,null,"Have appropriate inspection and acceptance test procedures been identified for prototype units?","(blank)"]
[null,null,"Have initial requirements been identified for acceptance test procedures and in-process and final inspection requirements for EMD units?","(blank)"]
[null,7,"Are control plans completed for management of Key Characteristics?","(blank)"]
[null,null,"Are Test and Inspection Plans being developed for EMD units?","(blank)"]
[null,null,"Has quality data from the production representative environment been collected and analyzed and the results used to shape improvement plans?","(blank)"]
[null,8,"Are Key Characteristics managed with measurement procedures and controls in place (e.g. SPC, FRACAS, audits, customer satisfaction, etc.)?","(blank)"]
[null,null,"Are Test and Inspection Plans complete and validated for production units?","(blank)"]
[null,null,"Does the Pilot line data meet capability requirements for all Key Characteristics?","(blank)"]
[null,9,"Does data from LRIP demonstrate that production processes for all Key Characteristics and other manufacturing processes critical to quality are capable and under control for FRP?","(blank)"]
[null,10,"Do results of quality data achieve targeted statistical level on all Key Characteristics?","(blank)"]
[null,null,"Do results of quality data reflect continuous improvement?","(blank)"]
[null,null,"Is the quality of Key Characteristics controlled at rate, and the data reflects only rare and unrepeated quality issues related to manufacturing processes?","(blank)"]
["F.3 - Supplier Quality Management",4,"Are potential supplier base quality capabilities and risks identified, to include subtier supplier quality management? ","(blank)"]
[null,5,"Have supply base quality capabilities and risks been identified, to include subtier quality management?","(blank)"]
[null,6,"Have specific supply base quality improvement initiatives been identified addressing supplier Quality Management System shortfalls, including subtier supplier quality management?","(blank)"]
[null,7,"Do key supplier Quality Management Systems meet appropriate industry standards? ","(blank)"]
[null,null,"Has a strategy been outlined for audits of critical supplier processes?","(blank)"]
[null,null,"Has supplier quality data from production representative units been collected and analyzed?","(blank)"]
[null,8,"Are supplier program-specific Quality Management Systems adequate ?","(blank)"]
[null,null,"Have supplier products completed first article inspection?","(blank)"]
[null,null,"Have supplier products completed qualification testing?","(blank)"]
[null,null,"Is a plan for subcontractor process audits in place and being implemented by prime contractor?","(blank)"]
[null,null,"Is acceptance testing of supplier products adequate to begin LRIP?","(blank)"]
[null,9,"Are subcontractor Quality Audits performed as necessary to ensure subcontractor specification compliance?","(blank)"]
[null,null,"Does acceptance testing of supplier products reflects control of quality adequate to begin FRP?","(blank)"]
[null,null,"Does supplier management of quality of Key Characteristics and other critical manufacturing processes demonstrate capability and control for FRP?","(blank)"]
[null,10,"Are subcontractor Quality Audits performed as necessary to ensure subcontractor specification compliance?","(blank)"]
[null,null,"Does process quality achieve high statistical level (e.g. 6-sigma) on all critical dimensions? ","(blank)"]
[null,null,"Does supplier quality data reflect adequate management of Key Characteristics and control of critical manufacturing processes, including quality management down to subtier suppliers?","(blank)"]
["G - Mfg Personnel"]
["G.1 - Manufacturing Personnel",3,"Have new manufacturing skills been identified?","(blank)"]
[null,null,"Have the manufacturing skills needed to produce, test and support the proposed concepts been assessed?  ","(blank)"]
[null,4,"Has the availability of process development workforce for the Technology Maturation and Risk Reduction Phase been determined?","(blank)"]
[null,null,"Have manufacturing skill sets been identified?","(blank)"]
[null,null,"Have production workforce requirements (technical and operational) been evaluated as part of AoA?","(blank)"]
[null,5,"Have plans for the identified skill sets been developed to meet prototype production needs?","(blank)"]
[null,null,"Have special skills certification and training requirements been established?","(blank)"]
[null,null,"Have the necessary workforce skill sets been identified to meet prototype production needs?","(blank)"]
[null,6,"Are the manufacturing workforce skills available for production in a relevant environment?","(blank)"]
[null,null,"Have initial plans for the identified resources (quantities and skill sets) been developed to achieve requirements for pilot line and production? ","(blank)"]
[null,null,"Have resources (quantities and skill sets) been identified to achieve requirements for pilot line and production? ","(blank)"]
[null,7," Have plans been developed to achieve pilot line requirements?","(blank)"]
[null,null," Have plans been updated to achieve Low Rate Initial Production (LRIP) workforce requirements? ","(blank)"]
[null,null,"Has the pilot line workforce been trained in the representative environment?","(blank)"]
[null,null,"Have the manufacturing workforce resource requirements been identified for the pilot line?","(blank)"]
[null,8," Have plans been developed to achieve Low Rate Initial Production (LRIP) requirements?","(blank)"]
[null,null," Have plans been updated to achieve  Full Rate Production (FRP) workforce requirements? ","(blank)"]
[null,null,"Have Low Rate Initial Production (LRIP) personnel been trained on the pilot line where possible?","(blank)"]
[null,null,"Have manufacturing workforce resource requirements been identified for Low Rate Initial Production (LRIP)?","(blank)"]
[null,9,"Has a plan been implemented to achieve Full Rate Production (FRP) workforce requirements?","(blank)"]
[null,null,"Have Low Rate Initial Production (LRIP) personnel requirements been met?","(blank)"]
[null,10,"Are production workforce skill sets being maintained due to attrition of the workforce?","(blank)"]
[null,null,"Have Full Rate Production (FRP) personnel requirements been met including all necessary certifications?","(blank)"]
["H - Facilities"]
["H.1 - Tooling/ Special Test and Inspection Equipment (STE/SIE)",4,"Are tooling/Special Test Equipment (STE)/Special Inspection Equipment (SIE) requirements being considered as part of AoA?","(blank)"]
[null,5,"Have supporting rationale and a schedule been provided for the tooling and STE/SIE requirements? ","(blank)"]
[null,null,"Have tooling and STE/SIE requirements been identified?","(blank)"]
[null,6,"Have production tooling and STE/SIE requirements been developed?","(blank)"]
[null,null,"Have prototype tooling and STE/SIE concepts been demonstrated in production relevant environment? ","(blank)"]
[null,7,"Are production tooling and STE/SIE design and development efforts underway?","(blank)"]
[null,null,"Has a manufacturing equipment maintenance strategy been developed?","(blank)"]
[null,8,"Has manufacturing equipment maintenance been demonstrated on pilot line?","(blank)"]
[null,null,"Have all tooling, test and inspection equipment requirements been identified for Low Rate Initial Production (LRIP)?","(blank)"]
[null,null,"Have all tooling, test, and inspection equipment been proven on the pilot line?","(blank)"]
[null,null,"Have all STE/SIE been validated as part of pilot line validation IAW validation plan?","(blank)"]
[null,9,"Has a manufacturing equipment maintenance schedule been demonstrated?","(blank)"]
[null,null,"Has all tooling, test, and inspection equipment been proven in LRIP and additional requirements identified for Full Rate Production (FRP)?","(blank)"]
[null,null,"Have all tooling, test, and inspection equipment been proven in the Low Rate Initial Production (LRIP)?","(blank)"]
[null,null,"Has STE/SIE validation been maintained as necessary? ","(blank)"]
[null,null,"Has STE/SIE revalidation been completed as necessary?","(blank)"]
[null,10,"Has a planned equipment maintenance schedule been achieved?","(blank)"]
[null,null,"Are proven tooling, test, and inspection equipment in place to support maximum Full Rate Production (FRP)?","(blank)"]
[null,null,"Has STE/SIE validation been maintained as necessary? ","(blank)"]
[null,null,"Has STE/SIE revalidation been completed as necessary?","(blank)"]
["H.2 - Facilities",3,"Have specialized facility requirements/needs been identified? ","(blank)"]
[null,4,"Has the availability of manufacturing facilities for prototype development and production been evaluated as part of the Analysis of Alternatives (AoA)?","(blank)"]
[null,5,"Have manufacturing facilities been identified to produce prototypes?","(blank)"]
[null,null,"Have plans for the identified manufacturing facilities been developed to produce prototypes?","(blank)"]
[null,6,"Have manufacturing facilities been identified to produce the pilot line build?","(blank)"]
[null,null,"Have plans for the identified manufacturing facilities been developed to produce the pilot line build?","(blank)"]
[null,7,"Have manufacturing facilities been identified to produce the Low Rate Initial Production (LRIP) build?","(blank)"]
[null,null,"Have plans for the identified manufacturing facilities been developed to produce the Low Rate Initial Production (LRIP) build?","(blank)"]
[null,8,"Are manufacturing facilities adequate to begin Low Rate Initial Production (LRIP)?","(blank)"]
[null,null,"Are plans in place to support the transition to Full Rate Production (FRP)?","(blank)"]
[null,null,"Have pilot line facilities been demonstrated?","(blank)"]
[null,null,"Is workplace safety adequate?","(blank)"]
[null,9,"Are capacity plans adequate to support Full Rate Production (FRP)? ","(blank)"]
[null,null,"Have manufacturing facilities been demonstrated in Low Rate Initial Production (LRIP)? ","(blank)"]
[null,10,"Are the production facilities in place to meet Full Rate Production (FRP) requirements?","(blank)"]
[null,null,"Has the capacity of the production facilities been demonstrated to meet Full Rate Production (FRP) requirements?","(blank)"]
["I - Mfg Management"]
["I.1 - Mfg Planning & Scheduling",4,"Has a manufacturing strategy been developed and integrated with an acquisition strategy?","(blank)"]
[null,null,"Have prototype schedule risk mitigation efforts been incorporated into the Acquisition Strategy (AS)?","(blank)"]
[null,5,"Has the manufacturing strategy been refined based upon the preferred concept?","(blank)"]
[null,null,"Have prototype schedule risk mitigation efforts been initiated?","(blank)"]
[null,6,"Are all system design related manufacturing events included in the Integrated Master Plan/Schedule (IMP/S)?","(blank)"]
[null,null,"Has the initial manufacturing approach been developed?","(blank)"]
[null,null,"Has the manufacturing risk mitigation approach for the pilot line or technology insertion programs been defined?","(blank)"]
[null,7,"Has manufacturing planning required to achieve MRL 8 been included in the Integrated Master Plan/Schedule (IMP/S)?","(blank)"]
[null,null,"Has the initial manufacturing plan been developed?","(blank)"]
[null,null,"Have initial work instructions been developed? ","(blank)"]
[null,null,"Have manufacturing risks been integrated into risk mitigation plans?","(blank)"]
[null,null,"Is an effective production control system in place to support the pilot line? ","(blank)"]
[null,8,"Has the manufacturing plan been updated for Low Rate Initial Production (LRIP)?","(blank)"]
[null,null,"Have all key manufacturing risks been identified and assessed with approved mitigation plans in place?","(blank)"]
[null,null,"Have work instructions been finalized? ","(blank)"]
[null,null,"Is an effective production control system in place to support Low Rate Initial Production (LRIP)?","(blank)"]
[null,9,"Has the manufacturing plan been updated for Full Rate Production (FRP)?","(blank)"]
[null,null,"Have all manufacturing risks been tracked and mitigated?","(blank)"]
[null,null,"Is an effective production control system in place to support Full Rate Production (FRP)?","(blank)"]
[null,10,"Have all manufacturing risks been mitigated?","(blank)"]
["I.2 - Materials Planning",4,"Has the technology development article component list been developed with associated lead time estimates?","(blank)"]
[null,5,"Do the make/buy evaluations include production considerations reflecting the pilot line, Low Rate Initial Production (LRIP), and Full Rate Production (FRP) needs?","(blank)"]
[null,null,"Have make/buy evaluations begun?","(blank)"]
[null,null,"Is the technology development part list maturing?","(blank)"]
[null,6,"Are most material decisions complete (make/buy)?","(blank)"]
[null,null,"Has the Bill of Materials (BOM) been initiated?","(blank)"]
[null,null,"Have material risk mitigation plans been developed?","(blank)"]
[null,null,"Have material risks been identified? ","(blank)"]
[null,7,"Are material planning systems in place for the pilot line build?","(blank)"]
[null,null,"Are the make/Buy decisions and Bill of Materials (BOM) complete for pilot line build?  ","(blank)"]
[null,8,"Are the Make/Buy decisions and Bill of Materials (BOM) complete to support Low Rate Initial Production (LRIP)?","(blank)"]
[null,null,"Have material planning systems been proven on the pilot line and are in place for Low Rate Initial Production (LRIP) build?","(blank)"]
[null,9,"Are the Make/Buy decisions and Bill of Materials (BOM) complete enough to support Full Rate Production (FRP)?","(blank)"]
[null,null,"Has the material planning system been proven in LRIP, and is it sufficient for Full Rate Production (FRP?","(blank)"]
[null,10,"Have the material planning systems been validated on the Full Rate Production (FRP) build?","(blank)"]