Stage 2 Preliminary Assessment - California

Stage 2 Preliminary AssessmentDepartment of Technology, SIMM 19B, Revision 7/1/2015

2.1 General InformationAgency or State Entity Name:Air Resources Board

Organization Code:

3900

Proposal Name:

Community Air Quality Monitoring Database and Web Portal

Department of Technology Project Number: 3900071

2.2 Preliminary Submittal InformationContact Information:

Contact First Name:Mena

Contact Last Name:Shah

Contact Email:[email protected]

Contact Phone Number:(916) 3247165

Preliminary Submission Date:

Preliminary Project Approval Executive Transmittal:File Attachment

2.3 Stage 2 Preliminary Assessment2.3.1 Impact Assessment Yes No

1. Has the Agency/state entity identified and committed subject matter experts from all business sponsors and key stakeholders?

2. Are all current baseline systems that will be impacted by this proposal documented and current (e.g., data classification and data exchange agreements, privacy impact assessments, design documents, data flow diagram, data dictionary, application code, architecture descriptions)?

3. Does the Agency/state entity anticipate needing support from the Department of Technology’s Statewide Technology Procurement Division to conduct market research for this proposal (Market Survey, Request for Information)?

4. Does the Agency/state entity anticipate submitting a budget request to support the procurement activities of this proposal?

5. Could this proposal involve the development and/or purchase of systems to support activities included in Financial Information System for California (FI$CAL) (e.g., financial accounting, asset management, human resources, procurement/ordering, inventory management, facilities management)?

mailto:[email protected]

6. Does the Agency/state entity have a designated Chief Architect or Enterprise Architect to lead the development of baseline and alternative solutions architecture descriptions?

7. Will the Agency/state entity’s Information Security Officer be involved in the development and review of any security related requirements?

8. Does the Agency/state anticipate performing a businessbased procurement to have vendors propose a solution?

2.3.2 Business Complexity Assessment

Business Complexity: 1.9 Business Complexity Zone: High Medium Low

Stage 2 Alternative Analysis2.4 Submittal Information

Contact Information: (Use Contact Information from Preliminary Submittal Information )

Contact First Name:Mena

Contact Last Name:Shah

Contact Email:[email protected]

Contact Phone Number:(916) 3247165

Submission Date:

Submission Type:

New Submission

Updated Submission (PreApproval)

Updated Submission (PostApproval)

Withdraw Submission

Project Approval Executive Transmittal:File Attachment

Condition(s) from Previous Stage(s):

Condition #

Condition Category

Condition SubCategory

Condition

Assessment Select or type...

Agency/state Entity Response

Status Select or type...

2.5 Baseline Processes and Systems2.5.1 DescriptionAir quality monitoring data underpin all programs at the California Air Resources Board (CARB). First and foremost,

mailto:[email protected]

ambient air monitoring data signal the location, type, and extent of air quality problems present throughout the State. Second, monitored data evaluate and improve the performance of tools used in the planning process. Lastly, air quality data act as metrics by which the State can measure progress and demonstrate program success.

CARB has a long history of monitoring and displaying criteria pollutants and toxics air contaminants. Since the 1960s CARB, air districts, and other entities have been measuring and storing air quality data, and CARB has been displaying the air quality data online since 2001.

CARB, air districts, and other data providers operate nearly 200 air quality monitors throughout the State. Through standard monitoring protocols and data quality objectives, data providers maintain sites and instruments so that the data collected are accurate and representative. Data providers postprocess data to ensure quality, and then submit official data to the United States Environmental Protection Agency’s (USEPA) Air Quality System (AQS) database. Official air quality data from AQS propagate to CARB’s Aerometric Data Analysis and Management (ADAM) system. These data are then summarized and ADAM calculates regulatory statistics to determine whether or not regions meet the level of the National Ambient Air Quality Standards (NAAQS). Many data providers also choose to submit realtime preliminary data to CARB’s Air Quality and Meteorological Information System (AQMIS). Near realtime preliminary data go into AQMIS’s prelim database, where they are also processed (parsed, go through quality assurance tests, and stored). Then they are sent to AQMIS’s merged database. Here, the quality assured preliminary data get combined with official meteorological data from CARB’s met database and official air quality data from ADAM. Once in merged, data are available to users via AQMIS’s public facing website. AQMIS web capabilities include tabular data displays, data query tools, graphing and mapping capabilities, and a mobile website (Breath Well) Official data, regulatory statistics, and toxics data are available through ADAM’s public facing web pages as well.

Historically, CARB, air districts, and other entities focused on measuring ambient criteria pollutant and toxic air contaminant data for federal regulatory purposes. In other words, these data came from monitors meeting particular siting requirements (e.g., not within a certain distance of a known pollution source), instrument requirements (e.g., monitors must be federally approved), and data quality requirements (e.g., follow calibration and audit procedures, data completeness, etc.).

AB 617 augments air quality monitoring by shifting focus to monitoring at the community level. The bill requires an air district containing a selected location, by July 1st, 2019, to deploy a community air monitoring system in that location. The bill authorizes the air district to require a stationary source that emits air pollutants in, or that materially effect, the selected location to deploy a fenceline monitoring system or other specified realtime onsite monitoring.

While AQMIS has near realtime capabilities for criteria pollutants, and ADAM contains toxic pollutants and regulatory statistics for criteria pollutants, a new database and web portal are critical to the success of the community air monitoring program. The current databases’ architecture (data streams, processing, quality assurance and quality management procedures, etc.) does not support the volume and type of the high resolution data from new sensor technology. Further, AQMIS and ADAM contain regulatory data comparable to the NAAQS, while the new community monitoring database will contain finer resolution, nonregulatory data that are not directly comparable to the NAAQS.

CARB divisions have extensive experience in community air monitoring, and have historically partnered with local air districts, communities, and academics to share information, expertise, and resources.

Currently, CARB is collaborating with US EPA’s Advanced Monitoring Workgroup, which includes SCAQMD’s AQSPEC Lab, to assess the feasibility of a sensor certification program and develop a scan and screen network to rapidly assess new technologies. SCAQMD’s AQSPEC Lab conducts laboratory and field evaluations and CARB will build on their experience to develop our inhouse capabilities, expanding California’s capacity to conduct evaluations of commercial products.

In 2016, CARB worked closely with the IVAN network in the Imperial Valley to compare air sensor performance to regulatorygrade instruments. The calibration algorithms developed from this comparison were used to calibrate all of the community sensors currently in operation. CARB’s Office of Emergency Response is currently evaluating

a suite of PM monitors including lowcost air sensors and “midtier” equipment in a smoke chamber (under contract) and in the field to compare performance and assess effectiveness for monitoring community wildfire smoke impacts.

In the near term future, CARB’s 201718 annual research plan includes $400,000 for developing realtime, portable monitoring methods for toxic metals. And MLD is sponsoring the upcoming “Air Sensors International Conference” in September 2018 with UC Davis and USEPA to bring together stakeholders from academia, government, communities, and commercial interests to promote and advance air pollution sensors, improve the data quality from these sensors, expand the pollutants measured, and foster community involvement in monitoring air quality.

The community air quality monitoring database and web portal will play a pivotal role in fulfilling requirements of AB 617. Additionally, this product will help build upon the relationship between communities, air districts, and CARB. Air districts and communities will submit data to CARB, and be able to see their data on the website in real time. The data transparency inherent to this program will allow a community to see clearly the cause and effect relationships. Visualization tools within the web portal will illustrate the air quality problems within a community and demonstrate progress that results from the community emissions reduction programs.

2.5.2 Business Process Workflow

3900_CARB_CAQM_BusinessProcessWorkflow.pdfAdobe Acrobat Document116 KB

2.5.3 Current Architecture Information

Business Function/Process(es)Webbased Real Time Air Quality and Meteorological Database

Application, System or Component:

AQMIS

COTS, MOTS or Custom: Custom ApplicationName/Primary Technology:

Postgresql, PHP, Apache

Runtime EnvironmentCloud Computing Used? Yes No

If "Yes", Specify: Platform as a Service (PaaS)

Server/Device Function: Database and Web ApplicationHardware: Dell (virtual)Operating System: Ubuntu 14.04.05 LTSSystem Software: Postgresql 9.3.14

Apache v2.4.7

PHP v5.5.9

System Interfaces: Custom web interfaceData Center Location: State Data Center Operated by Department of Technology

SecurityAccess:(check all that apply)

Public Internal State Staff External State Staff

Other, specify:

Type of Information:(check all that apply)

Personal Health Tax Financial Legal Confidential

Other, specify: air quality and meteorological data

Protective Measures:(check all that apply)

Technical Security Identity Authorization and Authentication Physical Security Backup and Recovery

Other, specify:

Data ManagementData Owner Name: Mena Shah

Title: AR Sup IBusiness Program: Air Quality and Statistical Studies Section

Data Custodian Name: Skip CampbellTitle: DPM IVBusiness Program: Systems Development and Support Branch

Business Function/Process(es)Webbased Official Air Quality and Toxics Database


ADAM

COTS, MOTS or Custom: Custom ApplicationName/Primary Technology:

Postgresql, PHP, Apache


If "Yes", Specify: Platform as a Service (PaaS)

Server/Device Function: databaseHardware: Dell (virtual)Operating System: Linux Red Htt 6.9System Software: Postgresql 9.6.5

Apache v2.2.5

PHP v5.3.3

System Interfaces: Custom web applicationData Center Location: State Data Center Operated by Department of Technology



Other, specify: Type of Information:(check all that apply)


Other, specify: air quality and toxics data

Protective Measures:(check all that apply)

Technical Security Identity Authorization and Authentication Physical Security Backup and Recovery

Other, specify:

Data Management

Data Owner Name: Mena ShahTitle: AR Sup IBusiness Program: Air Quality and Statistical Studies Section


2.5.4 Current Architecture Diagram

3900_CAQM_CurrentArchitectureDiagram.pdfAdobe Acrobat Document370 KB

2.5.5 Security Categorization Impact Table

3900_CARB_CAQM_SecurityImpactCategorizationTable.docxMicrosoft Word Document25.9 KB

SECURITY CATEGORIZATION IMPACT TABLE SUMMARYSECURITY OBJECTIVE

LOW MODERATE HIGH

Confidentiality

Integrity

Availability

2.6 MidLevel Solution Requirements

Requirements:

3900_CARB_CAQM_Midlevel_Requirements.xlsmMicrosoft Excel MacroEnabled Worksheet317 KB

2.7 Assumptions and ConstraintsAssumptions/Constraints Description/Potential Impact

Assumption: Program staff will be available. There will be adequate staff available fromthe Air Quality Planning and Science Division to define the vision/scope, requirements, and design; and to support the development, testing, and deployment of the application. Inadequate staffing will impact schedule and quality.

Assumption: Office of Information Services staff will beavailable

There will be adequate Server and Networking staffavailable to provide services as required. Unavailability will impact schedule.

Assumption: Dedicated staff will remain in their currentroles.

Staff assigned to the project will remain in theircurrent roles. Reassignment of staff will impactschedule.

Assumption: Project funding will be approved andavailable.

The project budget has been approved and will remain available throughout the project lifecycle. Funding nonavailability will impact quality, schedule, and resources.

Assumption: Current systems will remain available. The current systems will remain in production andmaintained while the new system is broughtinto production. Failure to do so will impact CARB's ability to provide emission inventories in support of multiple state and federal mandates.

Assumption: All base hardware/software requirementswill be met with existing CARB infrastructure.

All base hardware/software requirements will bemet with existing CARB infrastructure.

Assumption: Training Program will provide personnel for “Train thetrainer” sessions and will the provide training toprogram staff.

Constraint: Scope. The scope of the project is constrained to the Air Quality Planning and Science Division. Expanding scope will impact schedule and budget.

Assumption: contracted work. It is assumed that services will be procuredto develop and implement the chosen solution asrequired. Any required training is also included in this assumption. It is anticipated that a Request for Offer (RFO) will be released for prospective service providers to bid on.

Constraint: Hard deadline to procure services Vendor services must be procured before May 31, 2018 due to funding restrictions.

2.8 DependenciesElement Description

Resource Expertise Dependent on Program SME and technical experts being available

Project Approval Dependent on Department of Technology approval through PAL process

2.9 Market Research2.9.1 Market Research Methodologies/Timeframes

Methodologies used to perform market research (check all that apply):

Request for Information (RFI) Trade shows Internet Research Published Literature

Vendor Forums/Presentation Leveraged Agreements

Collaboration with other Agencies/state entities or governmental entities

Other, specify:

Collaborated with other Division performing similar activ...Time spent conducting market research: 3 monthsDate market research was started: 9/15/2017

Date all market research was completed: 11/17/2017

2.9.2 Results of Market ResearchThe California Air Resources Board (CARB) must develop a community air quality monitoring database and web portal to meet ongoing program needs and support new regulatory requirements. Specifically, Assembly Bill 617 (Garcia, 2017; AB 617) calls for improved multipollutant air pollution assessments at the community level, including an enhanced and targeted air quality monitoring program. Further, AB 617 requires CARB to publish the community monitoring data on its internet web site. The high resolution data from the community monitoring will need to be stored, quality assured, interpreted, and incorporated into CARB’s and air districts’ programs and further disseminated to the public. Therefore, CARB must create a repository for the data along with the web portal to allow public access to the data, and the visualization tools that present the data in meaningful and easytounderstand formats.

As part of the market research for the development of the CARB community air quality monitoring database and web portal, CARB staff performed an assessment of available commercial offtheshelf (COTS) solutions. While existing software is commercially available to address some aspects of the community monitoring database and web portal development, these tend to address only very specific and isolated functionality for the overall process that CARB must maintain. It was concluded through this market research that using COTS solutions would require extensive customization and integration of multiple software components jeopardizing the project schedule and level of quality needed for the implemented solution, with a custom solution from vendor being the best approach.

Due to the highly expedited timeframe set by AB 617 and requirements for encumbering the funding already authorized by Assembly Bill 109 (Ting, Budget Act of 2017), a formal Request for Information (RFI) process was not possible. However, CARB and other stakeholders are aware of several vendors that have expertise in developing realtime databases and web portalrelated components through past projects. These vendors were informed that CARB would be developing the community air quality monitoring database and web portal to address existing and upcoming program requirements. Through vendor demonstrations and highlevel discussions, it was determined that these vendors could meet some or all of the requirements for developing a community monitoring database and portal. A Request for Offer will be sent to the pool of vendors who have demonstrated experience in addressing the anticipated needs of the new community air quality monitoring database and web portal.

See attachment 3900_CARB_CAQM_Market_Research_Report for a detailed descrition of the market reasearch analysis completed.

2.10 Alternative Solutions2.10.1 Solution Type

Recommended Alternative

2.10.2 NameCustom Developed Replacement Solution Vendor

2.10.3 DescriptionEngage an external vendor to develop and implement a custom solution to 1) design and build the community air quality monitoring database), 2) design and build methods by which the database can ingest a variety of data types from a wide variety of sensors, and 3) build a comprehensive web portal complete with visualization tools (including mapping and graphing capabilities) and data query and tabular display of data capabilities. This solution will be designed to enable the Division to configure the system to meet changing business needs with minimal or

no involvement from the external vendor.

CARB will obtain through the procurement effort an external solution integration vendor experienced with software product development using the Waterfall methodology. CARB’s goal is to have a fully developed product within 12 months of project initiation.

The solution will be implemented through collaborative efforts of the integration vendor, CARB business staff and CARB information technology staff. The custom developed system will be hosted at the CA State data center in tenant managed services (TMS). Operations and maintenance of the custom system will be the responsibility of CARB business and IT resources.

Approach (check all that apply) Increase staff new or existing capabilities Modify the existing business process or create a new business process

Reduce the services or level of services provided Utilize new or increased contracted services

Enhance the existing IT system Create a new IT system

Perform a businessbased procurement to have vendors propose a solution

Other, specify:

2.10.4 Benefit AnalysisBenefits/Advantages

A custom solution meets all of the unique regulatory and functional requirements. The requirements for this application are unique to the program, making detailed customization necessary for the solution.

The custom solution could provide the Department greater control over how certain business processesare supported and accommodated within the solution.

The solution could be built to be adaptable to changing business needs.

There are no extraneous functions that would be included but remain unused, potentially affecting the speed and stability of the application

Uses CARBsupported technologies: a custom solution can be built using technologies and architecturesupported by CARB staff.

A custom solution will have greater consistency overall, rather than relying on the successful integration of many individual COTS/MOTS based components.

A custom solution can be built entirely with nonproprietary software components unlike a COTS/MOTS based solution. This facilitates the transparency of process and methodology required of a government regulatory agency.

A custom solution by a vendor approach will provide the opportunity to establish a rigid timeline for project completion and agreedupon scope consistent with the requirements for supporting AB617.

Disadvantages

The time commitment from business staff is substantial during the software development and stabilization

timeframe.

Custom software typically has numerous defects for the first one to three years until the software hasbeen stabilized.

Implementation cost: this solution would have a higher implementation cost, since it requires building acustom solution building a new system, as well as incorporating a new advanced data visualization portal.

Anticipated Time to Achieve Objectives After Project GoLiveObjective Number

Within 1 Year 2 Years 3 Years 4 Years Over 4 Years

1.1

1.2

2.1

2.2

Anticipated Time to Achieve Financial Benefits After Project GoLive

Financial Benefit Within 1 Year 2 Years 3 Years 4 Years Over 4 Years

Increased Revenues

Cost Savings

Cost Avoidance

Cost Recovery

2.10.5 Assumptions and ConstraintsAssumptions:1. OIS staff will support maintenance and operations for the new system, requiring additional staffing.

Constraints:1. Vendor services will be required to support the development and implementation of the custom solution.2. Schedule: It is anticipated that this effort will be constrained to 12 months. This estimate is based onsimilar CARB efforts in the past.3. Scope: The scope will be constrained to building a new community air quality monitoring system, with additional data display as directed AB617.4. System will be built using the current CARBsupported information technology architecture and services.

2.10.6 Implementation ApproachIdentify the type of existing IT system enhancement or new system proposed (check all that apply):

Enhance the current system Develop a new custom solution

Purchase a Commercial offtheShelf (COTS) system

Purchase or obtain a system from another government agency (Transfer)

Subscribe to a Software as a Service (Saas) system

Other, specify: Identify cloud services to be leveraged (check all that apply):

Software as a Service (SaaS) provided by OTech

Software as a Service (SaaS) provided by commercial vendor Platform as a Service (PaaS) provided by OTech

Platform as a Service (PaaS) provided by commercial vendor

Infrastructure as a Service (IaaS) provided by OTech

Infrastructure as a Service (IaaS) provided by commercial vendor

No cloud services will be leveraged by this alternative. Provide a description of why cloud services are not being leveraged:

CARB maintains a robust virtualized server environment at the CA State Data Center Tenant Managed Servicesfacility, allowing CARB to host this solution without additional hardware costs.

Identify who will modify the existing system or create the new system (check all that apply) Agency/state entity IT staff A vendor will be contracted

Interagency agreement will be established with another governmental agency. Specify Agency name(s):

Other, specify:

Identify the implementation strategy:

All requirements will be addressed in this proposed project in a single implementation.

Requirements will be addressed in incremental implementations in this proposed project.

Some requirements will be addressed in this proposed project. The remaining requirements will be addressed at a later date.

Specify the year when remaining requirements will be addressed: Identify if the technology for the proposed project will be mission critical and public facing:

The technology implemented for this proposed project will be considered mission critical and public facing.

2.10.7 Architecture Information

Business Function/Process(es)Community Air Quality Monitoring Database and Web Portal


CAQM System

COTS, MOTS or Custom: Custom Application

Name/Primary Technology: Postgresql, PHP, Python, Apache


If "Yes", specify: Platform as a Service (PaaS)

Server/Device Function: DatabaseHardware: Virtual ServerOperating System: LinuxSystem Software: Postgresql, PHP, Python, Apache

System Interfaces: Web applicationData Center Location: State Data Center Operated by Department of Technology

Security

Access:(check all that apply)




Other, specify: air quality, meteorological, and toxics data

Protective Measures: (check all that apply)

Technical SecurityPhysical Security

Identity Authorization and AuthenticationBackup and Recovery

Other, specify: Data ManagementData Owner Name: Mena Shah


Data Custodian Name: Skip CampbellTitle: DPM IVBusiness Program: System Development and Support Branch

2.10.1 Solution TypeRecommended Alternative

2.10.2 NameCustom Developed Replacement Solution InHouse

2.10.3 DescriptionUse internal CARB staff to develop and implement a custom solution to 1) design and build the community air quality monitoring database), 2) design and build methods by which the database can ingest a variety of data types from a wide variety of sensors, and 3) build a comprehensive web portal complete with visualization tools (including mapping and graphing capabilities) and data query and tabular display of data capabilities. This solution will be designed to enable the Division to configure the system to meet changing business needs with minimal or no involvement from the external vendor.

CARB will hire additional IT and business staff with experience in software product development using the Waterfall methodology and has estimated 1) dedicating a higher percentage of key staff time to participate in the development, testing, and deployment of the system components and 2) the addition of temporary resources to ensure the business is fully supported during the development period. With inhouse development, CARB’s goal is to have a fully developed product within 12 months of project initiation.

The solution will be implemented through collaborative efforts CARB business staff and CARB information technology staff. The custom developed system will be hosted at the CA State data center in tenant managed services (TMS).


Reduce the services or level of services provided

Utilize new or increased contracted services



Other, specify:


The solution could meet all of the unique regulatory and functional requirements: the requirements for this application are unique to the program, making detailed customization necessary for the solution.

The custom solution could provide the Department greater control over how certain business processesare supported and accommodated within the solution.

The solution could be built to be adaptable to changing business needs.

There are no extraneous functions that would be included but remain unused, potentially affecting the speed and stability of the application

Uses CARBsupported technologies: a custom solution can be built using technologies and architecturesupported by CARB staff.

Disadvantages

The time commitment from business staff is substantial during the software development and stabilizationtimeframe.

More internal IT staff will be required to support the development and stabilization of the software than wouldbe with either a vendor led custom development or a COTS/MOTS implementation.

There is a longer time to deployment than a vendor led custom development since the solution will need to beconstructed and tested over a longer period of time with fewer resources before being configured andimplemented.

If inhouse staff do not have adequate system development expertise, the attempt to build the system may fail;a low quality system may be implemented; and/or the user interface may be cumbersome to use.

Custom software typically has numerous defects for the first one to three years until the software has beenstabilized.



1.1

1.2

2.1

2.2



Increased Revenues

Cost Savings

Cost Avoidance

Cost Recovery

2.10.5 Assumptions and ConstraintsAssumptions:1. The solution can be developed through a traditional Waterfall, Agile, or other iterative development methodology.2. OIS staff will support maintenance and operations for the new system. This may require additional staffing.

Constraints:1. If an Agile methodology is used, the CARB program and IT staffs will need to make a more significant timecommitment and more staff may be required to complete ongoing work.2. Additional IT staff will need to be hired to support development and stabilization.3. Additional training of inhouse IT staff will be required to increase software development expertise onAgile methodology, selected development platform, security, and data management.4. Speed of system development is entirely dependent on Department priorities, adherence to schedule, andurgency for completion.5. Schedule: It is anticipated that this effort will be constrained to 12 months. This estimate is based onsilmilar CARB efforts in the past.6. Scope: The scope will be constrained to building a new community air quality monitoring system, with additional data display as directed AB617.7. System will be built using the current CARBsupported information technology architecture and services.


Enhance the current system Develop a new custom solution

Purchase a Commercial offtheShelf (COTS) system

Purchase or obtain a system from another government agency (Transfer)

Subscribe to a Software as a Service (Saas) system


Software as a Service (SaaS) provided by OTech

Software as a Service (SaaS) provided by commercial vendor Platform as a Service (PaaS) provided by OTech

Platform as a Service (PaaS) provided by commercial vendor




CARB maintains a robust virtualized server environment at the CA State Data Center Tenant Managed Servicesfacility, allowing CARB to host this solution without additional hardware costs.

Identify who will modify the existing system or create the new system (check all that apply) Agency/state entity IT staff

A vendor will be contracted


Other, specify:










CAQM System

COTS, MOTS or Custom: Custom Application

Name/Primary Technology: Python/ .NET


If "Yes", specify: Platform as a Service (PaaS)

Server/Device Function: DatabaseHardware: Virtual ServerOperating System: Microsoft Server 2012System Software: Microsoft SQL Server 2016

System Interfaces: Web applicationData Center Location: State Data Center Operated by Department of Technology





Other, specify: air quality, toxics, and meteorological data







2.10.1 Solution TypeRecommended Alternative

2.10.2 NameModified Off The Shelf (MOTS)

2.10.3 DescriptionEngage an external vendor to configure, modify, integrate, and implement an integrated Modified Off the Shelf(MOTS) solution to 1) design and build the community air quality monitoring database), 2) design and build methods by which the database can ingest a variety of data types from a wide variety of sensors, and 3) build a comprehensive web portal complete with visualization tools (including mapping and graphing capabilities) and data query and tabular display of data capabilities.

CARB will select through the procurement effort a MOTS product and an external solution integration vendor specifically experienced with integrating that product. However, because of the unique air monitoring database and web portal requirements, it will be necessary for the system integration vendor to not only configure the COTS product but to also perform extensive software customizations. Depending on the level of customization available for the product procured, these software customizations may take the form of custom secondary scripts within the software product. Customization may also include developing custom software outside of the COTS product that performs operations on the COTS database or a custom developed database where data is replicated between it and the COTS database.

The solution will be configured, modified, and implemented through collaborative efforts of the integrationvendor, MOTS product vendor (where applicable), CARB business staff, and CARB information technology staff.The MOTS may be hosted by the integration vendor and implemented as a Software as a Service (SaaS) andinclude a Platform as a Service (PaaS) service to enable ongoing development and maintenance. Providing PaaSservices as part of the solution will enable Agency to configure the system to meet changing business needs withminimal or no additional software development or involvement from the external vendor. Operations andmaintenance of the system customizations will be the responsibility of the systems integration vendor and CARBbusiness and IT resources. An ongoing per user subscription is required to use and obtain support for the product.


Reduce the services or level of services provided Utilize new or increased contracted services



Other, specify:


MOTS software products are typically more stable and mature because of each MOTS company’s dedication tosoftware development; adherence to formal institutionalized processes; 100% dedication of knowledge teams to

the business domain; and continual vetting and refining of their products over years with multiple similarcustomers.

There is a shorter time to deployment than a custom developed solution since the reviewed MOTS products willmeet the majority of the requirements with low levels of software development required.

The time commitment from business staff is minimal to moderate during the software configuration,customization, and deployment timeframe.

MOTS software products are typically based on technology platforms that enable the exchange of data with avariety of other technology platforms and formats, and allow vendors to customize, extend, test, and buildapplications and workflows to meet specific needs.

The modular nature of the known MOTS products will allow the Agency the flexibility for a variety of phasedimplementation approaches.

MOTS software may have additional capabilities that can be exploited when required at a later date, generallyfor a relatively small increase in software license cost or in some cases for no additional cost.

MOTS software is typically being continually improved, with additional functionality and technicalenhancements being added on a frequent basis.

Resources for knowledge of and expertise in the potential MOTS software packages are available throughmultiple vendors, providing options for implementation, support, and maintenance.

Knowledgeable customer support for MOTS software is typically stable through the years of ownership. Helpdesk support and training are available.

MOTS software is typically well documented, and the documentation is updated for each change to the software.

The initial price of a MOTS solution is substantially less than the initial cost to develop a custom solutionsince CARB benefits from the advantage of having development and support costs shared across customers thatpurchase the MOTS software products.

Disadvantages

CARB will likely need to adjust their business processes to align with the configuration of the MOTS solution, however the process is strictly constrained by database ans web portal visualization requirements.

The amount of business process change could impact the duration of implementation and staff acceptance oforganizational change.

Configuration of the MOTS solution to implement some of the requirements might require workarounds thatresult in a less than desirable interaction with the system.

Additional software development could be required to customize CARBspecific regulatory requirements.

There is a risk that the vendor could stop supporting the software in the future.

MOTS solutions may be based on proprietary software components that inhibit the complete transparency of methodologies required by a government regulatory agency.

A MOTS solution would likely require significant customization since the existing market is nonexistent for CARB's unique business case.



1.1

1.2

2.1

2.2



Increased Revenues

Cost Savings

Cost Avoidance

Cost Recovery

2.10.5 Assumptions and ConstraintsAssumptions:1. The selected MOTS software will continue to be a viable and supported product for the foreseeable future.2. Multiple vendors are available to configure, customize, and maintain a MOTS implementation.3. CARB is not limited to using CalCloud or other OTech services to support the implementation of theselected MOTS software.4. The MOTS software can be configured, customized, and/or extended to fully meet requirements.

Constraints:1. Vendor services will be required to support the development and implementation of the MOTS solution.2. Strategies for other Agency enterprise solutions (e.g., use of other COTS/MOTS products, overall enterprisearchitecture, data governance/management, etc.) need to be finalized before implementing any MOTSsolution.3. The product must be configured, customized, and/or extended to fully meet requirements.


Enhance the current system

Develop a new custom solution Purchase a Commercial offtheShelf (COTS) system

Purchase or obtain a system from another government agency (Transfer) Subscribe to a Software as a Service (Saas) system


Software as a Service (SaaS) provided by OTech Software as a Service (SaaS) provided by commercial vendor

Platform as a Service (PaaS) provided by OTech Platform as a Service (PaaS) provided by commercial vendor




Identify who will modify the existing system or create the new system (check all that apply) Agency/state entity IT staff A vendor will be contracted


Other, specify:










CAQM System

COTS, MOTS or Custom: Modified offtheshelf (MOTS)

Name/Primary Technology: Air Vision


If "Yes", specify: Select...

Server/Device Function: DatabaseHardware: Virtual ServerOperating System: Microsoft Server 2012System Software: Microsoft SQL Server 2016

System Interfaces: Web applicationsData Center Location: State Data Center Operated by Department of Technology





Other, specify: air quality, toxics, and meteorological data







2.11 Recommended Solution2.11.1 Rationale for SelectionA custom solution developed by a vendor allows CARB to meet the requirements of AB617 with complete user satisfaction and the ability to adapt to future business needs. While an inhouse developed solution has many of the same advantages as a vendor developed solution, the additional staffing and training requirements would put the development timeframe outside acceptable limits. A MOTS solution would also be less cost effective since no current commercial solution exists to comprehensively handle the unique needs of a flexible statewide community air quality monitoring system, and developing the needed functionality while omitting extraneous functionality would take more time and effort than the custom solution.

File Attachment

2.11.2 Technical/Initial CAPMM Complexity AssessmentComplexity Complexity Zone

Technical Complexity Score: 1.8Zone I Low Criticality/Risk

Zone II/III Medium Criticality/Risk

Zone IV High Criticality/Risk

2.11.3 Procurement and Staffing Strategy

ActivityCost Estimating

Responsible(check all that apply)

Agency/State Entity Staff DGS StaffSTPD Staff

ITPOD StaffCAPMO StaffContractor

Other, specify:

When Needed(check all that apply)

Stage 3 Solution Development Stage 4 Project Readiness and Approval

After project is approved (after Stage 4 Project Readiness and Approval)

Cost Estimate Verification(check all that apply)

Market research conducted (MR) Cost estimate provided (CE) Department of Technology CE

DGS CERequest For Information conducted (RFI)Comparable vendor services have been used on previous contracts (CV)Leveraged Procurement Agreement (LPA)

Complete Only if Contractor Responsible for Activity

Procurement VehicleSelect...If “Other,” specify: Cost Estimating will be conducted by Procurement Officer

Contract TypeSelect...If “Other,” specify:

ActivitySolicitation Development




Other, specify:








Procurement VehicleSelect...If “Other,” specify: Solicitation Development to be primary responsibility of Contract Analyst


ActivityConduct Procurement




Other, specify:


Stage 3 Solution DevelopmentStage 4 Project Readiness and ApprovalAfter project is approved (after Stage 4 Project Readiness and Approval)





Procurement VehicleSelect...If “Other,” specify: Procurement Officer


ActivityProject Management




Other, specify:





Market research conducted (MR) Cost estimate provided (CE)

Department of Technology CEDGS CERequest For Information conducted (RFI)

Comparable vendor services have been used on previous contracts (CV)Leveraged Procurement Agreement (LPA)


Procurement VehicleRequest for Offer/California Multiple Award Schedules (RFO/CMAS)If “Other,” specify: PM Duties split between PM and Contractor PM

Contract TypeTime and Materials (T&M)If “Other,” specify:

ActivityBusiness Analysis


Agency/State Entity Staff DGS Staff

ITPOD StaffCAPMO Staff

STPD Staff Contractor

Other, specify:









Procurement VehicleRequest for Offer/California Multiple Award Schedules (RFO/CMAS)If “Other,” specify: Split between contractor BA and state BA


ActivityTechnical Analysis




Other, specify:









Procurement VehicleRequest for Offer/California Multiple Award Schedules (RFO/CMAS)If “Other,” specify: Split between Contractor Architect and State SSSIII


Activity

Requirements Elicitation




Other, specify:









Procurement VehicleRequest for Offer/California Multiple Award Schedules (RFO/CMAS)If “Other,” specify: Split between contractor BA and state BA


ActivityOrganizational Change Management




Other, specify:





Department of Technology CEDGS CERequest For Information conducted (RFI)Comparable vendor services have been used on previous contracts (CV)Leveraged Procurement Agreement (LPA)


Procurement VehicleSelect...

Contract TypeSelect...

If “Other,” specify: Air Resources Sup II

If “Other,” specify:

ActivityTesting




Other, specify:








Procurement VehicleRequest for Offer/California Multiple Award Schedules (RFO/CMAS)If “Other,” specify: Staff Air Pollution Specisalist/Air Resources Engineer


ActivityDesign




Other, specify:





Department of Technology CEDGS CERequest For Information conducted (RFI)Comparable vendor services have been used on previous contracts (CV)

Leveraged Procurement Agreement (LPA)Complete Only if Contractor Responsible for Activity

Procurement VehicleRequest for Offer/California Multiple Award Schedules (RFO/CMAS)If “Other,” specify: Contractor Architect/Staff Programmer Analyst


ActivityIntegration/Development




Other, specify:








Procurement VehicleRequest for Offer/California Multiple Award Schedules (RFO/CMAS)If “Other,” specify: Contractor Architect/Staff Programmer Analyst


ActivityData Cleansing




Other, specify:




Market research conducted (MR)Cost estimate provided (CE)




Procurement VehicleRequest for Offer/California Multiple Award Schedules (RFO/CMAS)If “Other,” specify: Contractor Data Analyst/Staff Programmer Analyst


ActivityData Validation




Other, specify:








Procurement VehicleRequest for Offer/California Multiple Award Schedules (RFO/CMAS)If “Other,” specify: Contractor Data Analyst/Staff Programmer Analyst/Staff Air Pollution Specialist


ActivityTraining




Other, specify:

When Needed Stage 3 Solution Development

(check all that apply) Stage 4 Project Readiness and ApprovalAfter project is approved (after Stage 4 Project Readiness and Approval)






Procurement VehicleRequest for Offer/California Multiple Award Schedules (RFO/CMAS)If “Other,” specify: Contractor BA/Air Resources Engineer


ActivityContract Management




Other, specify:







Procurement VehicleSelect...If “Other,” specify: State Contract Manager (DPM IV)


ActivityEnterprise Architecture




Other, specify:








Procurement VehicleSelect...If “Other,” specify: DPM IV


ActivityQuality Assurance




Other, specify:









Procurement VehicleRequest for Offer/California Multiple Award Schedules (RFO/CMAS)If “Other,” specify: Contractor QA/Air Resources Engineer


ActivityTechnical Installation of Hardware




Other, specify:







Procurement VehicleSelect...If “Other,” specify: SSSIII


ActivityTechnical Installation of Software




Other, specify:








Procurement VehicleRequest for Offer/California Multiple Award Schedules (RFO/CMAS)If “Other,” specify:


Contractor Developer/SSSIII

ActivityMaintenance




Other, specify:








Procurement VehicleRequest for Offer/California Multiple Award Schedules (RFO/CMAS)If “Other,” specify: Contractor/Staff Programmer Analyst


ActivityOperations




Other, specify:






Complete Only if Contractor Responsible for ActivityProcurement VehicleSelect...If “Other,” specify: All program staff


Yes NoWill any of the activities identified above result in a competitive or noncompetitive solicitation that will be over the Agency/state entity’s DGS delegated purchasing authority?

2.11.4 Enterprise Architecture AlignmentCARB has an existing Enterprise Architecture Framework consisting of a methodology, a conceptual framework anda process that will build on the foundations laid down in the overall IT strategy. The conceptual framework and methodology are based on the California Enterprise Architecture Framework (CEAF 2.0), The Open Group Architecture Framework (TOGAF). The Architecture Process provides for the creation of a vision and for the development of an architecture vision that realizes the IT Strategy. The vision will include business, application, data and technology architecture layers followed by migration planning, implementation governance and change management that works in conjunction with the project management process defined by CARB's Project Management Office.

Information Technology Capability TableInformation Technology Capability Existing Enterprise

Capability to be LeveragedNew Enterprise Capability

NeededPublic or Internal Portal/Website

Public or Internal Mobile Application

Enterprise Service Bus

Identity and Access Management

Enterprise Content Management (including document scanning and eForms capabilities)Business Intelligence and Data Warehousing

Master Data Management

Big Data Analytics

2.11.5 Project Phases

Phase

Phase 1. System ArchitectureDescription

This phase covers an assessment of AB617related systems in order to provide system architecture recommendations for developing the Community Monitoring database. Phase 1 task includes but not limited to the following:

1. Analyze and prioritize CARB's technical and functional requirements jointly with CARB subject matter experts to assess the emission inventory, air quality monitoring, and BBT systems, their associated output products, and anticipated data fields resulting from upcoming reporting requirements.

2. Design the software solutions and database architecture that meets the technical requirements collaboratively with CARB subject matter experts and program staff. CARB anticipates an iterative development process that will include a design document updated as needed, including wire frame page mockups, form and report specifications, business rules, batch process logic, application prototypes, database designs, logical and physical data models and system security.

3. Define and design data model and structures.

Phase 1 is projected approximately to take 3 months.Phase Deliverable

Database system architecture design artifacts and recommendations document .

Database Data Model and Structure Document.

Phase

Phase 2. Database Design and DevelopmentDescription

This phase covers the design of a complete community monitoring air quality system, including database and web portal. Phase 2 also includes development of the system's backend database. Phase 2 Tasks may include but are not limited to the following:

1. Final Project Management Plan Develop, maintain and deliver a written project plan entailing the project tasks, schedule, and expected level of effort required from both Contractor and CARB throughout the project. The project plan includes a high level work plan that includes tasks and a work breakdown structure for system development tasks; dependencies; key project deliverables; milestones; and critical path. The project plan also includes project controls that will be used to monitor and manage project progress and direction. Project controls include a communication strategy to obtain input and feedback on CARB’s business objectives and requirements from CARB staff. Report implementation progress in terms of specific metrics, including but not limited to completed, inprogress, and remaining deliverables; reports provided as deliverables for each subtask and the incorporation of feedback from CARB; and any impediments or risks to successful and timely completion of the Task 1 deliverables.

2. System Design Review and analyze CARB’s technical and functional requirements jointly with CARB subject matter experts and recommend clarifications and corrections. The system design task also includes defining a software solution, including proposed technology to meet the CAQM database requirements. CARB anticipates an iterative and collaborative solution development process that will have design artifacts created throughout the development period, such as wire frame page mockups, form and report specifications, business rules, batch process logic, application prototypes, and database designs, including logical and physical data models and system security. One requirement for the final system design is that is must be in software and technologies that is capable of being fully supported by CARB resources, including equipment and staff. It is also preferable if the system components are designed to use only open source software.

3. Requirements Traceability Matrix Development – Develop, review, and maintain a requirements traceability matrix to guide the building of the finished product during the development and testing activities. This review also includes prioritizing and mapping CARB’s business process requirements, and user perspective of the problem and solution. These requirements may include but are not limited to: data fields likely to be included in future data submission requirements, data fields necessary to communicate with other CARB databases including any related system being designed concurrently, and output summaries used by CARB staff.

4. Database Documentation Development – Develop the business and technical design documentation to design specifications. Review and obtain approval of design documentation with the CARB project manager and project management team.

5. Database Development – Translate the design specifications from Tasks 2.1 – 2.2 and 3.1 into system components. Code, comment, and unit test application components. CARB anticipates an iterative development process that will result in frequent delivery of working software for system integration and user acceptance testing. Deliver all source code iteratively throughout the development process via a Statehosted source code version control system. The database may include, but is not limited to tables with the following information for: parameter type, default units, data source/provider, quality, data

resolution, unit conversion, site/monitor locations, monitor type, etc. The database should also be flexible enough to allow for additional tables, fields within tables, as well as the addition of parameters, monitors, sites, etc. as the community monitoring program grows. The database should be designed so that it can ingest and store data from many different types of data sources including but not limited to stationary sites and mobile monitoring platforms.

6. Develop Data Inventory Routines – Develop routines and create tables within the database that store numbers regarding the data inventory. This may include, but is not limited to data counts by monitor and/or site, source, parameter, year, month, last date the data were updated, and hash updates,

7. Database Testing – Perform functional, integration, performance, and security testing of all programming modules and utilities for the new CAQM database to determine if the design meets the CARB business requirements. CARB anticipates the use of automated test tools for unit and integration testing (i.e., continuous integration). Track, document, and report all defects discovered and resolved defects during testing (include test reports in weekly status updates). Fix all reported defects; redeliver and retest all defect repairs. Support user acceptance testing by CARB’s subject matter experts.

8. Data Requirements Definition Define data descriptions, types, formats, and relational entity structures for storing these data in a relational database management system. These include but are not limited to; a standard format to store detailed records of criteria pollutants and toxic air contaminants air quality and related data submitted to the system, a metadata format which allows for effective tracking and organizing of air quality data, data structures providing a layer of compatibility with other CARB databases and visualization tools including any related systems being designed concurrently, and a mechanism to output data products in a format that supports the input requirements of subsequent database and modeling systems.

Phase 2 is estimated to take approximately 3 months.

Phase Deliverable

Final Project Management Plan

System Design Document

Requirements Traceability Matrix

Database Documentation

Data Inventory Routines Source Code and/or Software

A working CAQM database

Database Testing Results

Data Dictionary

Phase

Phase 3. Data Processing and Web Portal Design and DevelopmentDescription

This phase covers the design and build of data processing components as well as the design and build of CAQM web portal and visualization tools (frontend). Phase 3 also includes testing of the data processing components and web portal along with documentation. Phase 3 Tasks may include but are not limited to the following:

1. Data Processing Documentation Development – Develop the business and technical design documentation to design specifications for the data streaming and upload process, QA processes, and data averaging procedures. Review and obtain approval of design documentation with the CARB project manager and project management team.

2. Develop Methods for Real Time Data Streaming – Utilizing the data communications standards recommended and/or adopted by CARB’s Monitoring and Laboratory Division (MLD), develop and test multiple methods by which the realtime high resolution data from varying types of portable lowcost sensors enter into the CAQM database. While it may be one standardized method by which the realtime data stream into the database, the method must be compatible with many different kinds of sensors, and have the ability to be modified or enhanced with rapidly advancing technology.

3. Develop Methods for Regulatory Data Upload – Collaborate with CARB’s AQPSD staff to design and develop efficient methods by which air quality and meteorological data from current regulatory monitors stored in CARB’s Air Quality Meteorological Information System (AQMIS) and/or Aerometric Data Analysis and Management (ADAM) system can be uploaded into the CAQM database. These methods should not interfere with current AQMIS and ADAM processing loads and capabilities.

4. Develop Methods for Backfill Data Upload – Design and develop methods by which the high resolution sensor data and/or regulatory data may be loaded into the database in instances in which the methods developed in Task 4.1 and 4.2 may have the potential to fail (e.g., if the data upload stream fails, but sensors continue to collect data).

5. Develop Methods for Laboratory Data Upload – It is anticipated that some community air monitoring systems may include lower resolution data samples collected over longer time periods that need to analyzed with laboratory equipment and techniques (which may include but are not limited to: inductively coupled plasmamass spectrometer, high performance liquid chromatograph, gas chromatographmass spectrometer with thermal desorption). The contractor shall work with MLD and AQPSD staff to design and develop methods by which the discrete data are submitted and stored in the CAQM database.

6. Develop a Data Quality Assurance Infrastructure – A quality assurance infrastructure must be ubiquitous throughout the entire air quality monitoring system. There are several principles of quality assurance (QA) which will be relevant in different system components. Data integrity should be verified at every stage of the system, which includes checks for completeness (gaps or duplicates) and the presence of required key fields. Working with AQPSD, the Contractor shall develop automated QA routines that qualify all data (realtime and backfilled) using a threetiered system (1good quality observations, 2likely good quality, but some reason to doubt, and 3suspect observations). Predefined threshold values should be used to flag values that are either out of a reasonable range, or that change by an unusual amount compared to previous observations. The system should also contain a way in which the quality assigned through the automated QA can be overridden or changed manually.

7. Develop Automated Data Aggregation – Since regulatory methods exist by which criteria pollutant and toxic air contaminants are measured, it is important to have the capability to compare the high resolution data from low cost sensors with the lower resolution (hourly or daily) data from the federal reference method (FRM) and federal equivalent method (FEM) monitors. In consultation with CARB’s AQPSD, develop automated routines within the database that aggregate the high resolution sensor data into longerterm averages (hourly, daily, and even longerterm) so they can be compared to FRM/FEM monitors as well as the NAAQS for criteria pollutants.

8. Perform Processing Testing – Perform functional, integration, performance, and security testing of all method modules and utilities for the new database system to determine if the design meets CARB’s business requirements. CARB anticipates the use of automated test tools for testing. Track, document, and report all defects discovered and resolved defects during testing (include test reports in weekly status updates). Fix all reported defects; redeliver and retest all defect repairs.

9. Web Page Documentation Develop the business and technical design documentation to design specifications for the userfriendly web interface. Review and obtain approval of design documentation with the CARB project manager and project management team.

10. Web page Development Translate the design specifications from Tasks 2.1 – 2.2 and 5.1 into system components. Code, comment, and unit test application components. CARB anticipates an iterative development process that will result in frequent delivery of working software for system integration and user acceptance testing. Deliver all source code iteratively throughout the development process via a Statehosted source code version control system. The web page may include, but is not limited to the query tool, data and summary report download capabilities, and print capabilities. The user interface should be designed and developed to be a fully functional website but also have the capability to work on mobile devices (e.g., phones and tablets). Input from districts, community members, and other interested parties will be essential in determining these userinterface requirements.

11. Data Query Tool Development – The data query tool is the primary component through which most users will interact with the air quality monitoring database. It must provide standard and interactive summaries of the data in a userfriendly and flexible manner. The following are types of output products expected from the query tool: formal summaries with predefined formats with less interactivity, but with the ability to filter by region, year, and pollutant; and adhoc summaries which provide a more interactive resource to perform exploratory analysis, summarization, and quality assurance of the monitoring data.

12. Web Page Testing – Perform functional, integration, performance, and security testing of all web page modules and utilities for the new database system to determine if the design meets CARB’s business requirements. CARB anticipates the use of automated test tools for testing. Track, document, and report all defects discovered and resolved defects during testing (include test reports in weekly status updates). Fix all reported defects; redeliver and retest all defect repairs.

13. Web Portal Visualization Tools Documentation – Develop the business and technical design documentation to design specifications for the data visualization module within the web portal. Review and obtain approval of design documentation with the CARB project manager and project management team.

14. Determine Data Interpretation – In close consultation with CARB’s MLD, AQPSD, and Research Division (RD) staff, determine the methods for which to display data to the public in formats that are easy to understand and meaningful. The Contractor shall work closely with CARB to understand how data points are generated, what they mean, their spatial and temporal representativeness, relative uncertainty, potential source regions, data integrity, data accessibility, and appropriate usage. This may include but is not limited to: defining a colorcoded scale by which to categorize each criteria pollutant and toxic air contaminant; determining descriptive language regarding potential health effects and/or possible actions to take at various levels of concentrations.

15. Visualization Development – Translate design specifications from Tasks 2.12.2 and 6.1 into system components. Code, comment, and unit test application components. CARB anticipates an iterative development process that will result in frequent delivery of working software for system integration and user acceptance testing. Deliver all source code iteratively throughout the development process via a Statehosted source code version control system. The visualization module may include, but is not limited to: presenting data via a mapbased or other geographic information system (GIS) display with colorcoded concentrations on statewide and community scales; interactive timeseries graphs, and other possible data graphing capabilities determined in the future. The mapbased display should have user capabilities to pan and zoom, display more detailed information (e.g., site information, concentrations, etc.) when a monitor/concentration is selected, add and remove other layers of data (i.e., more or less pollutants), and control the date/time of concentrations to display in map (with default displaying the most recent observations). Time series plots should also be flexible so that user can determine the location, pollutant or toxics species, and time period (shortterm time series and longer term historical trends) for which to view data. Input from districts, community members, and other interested parties will be essential in determining these specific visualization requirements.

16. Integrate Data from Other Databases into Visualization – Develop a method by which data from the integrated emissions inventory and the technology clearinghouse databases may also be appropriately displayed within visualization tools. This may include but is not limited to incorporating emissions inventory data and technology data (e.g., technology implementation dates) as layers that can be toggled in the time series graphs; and locations of and emissions data from facilities in the mapbased display. Recommend any protocols and application programming interfaces (APIs) for communicating with other CARB databases.

17. Visualization Testing – Perform functional, integration, performance, and security testing of all

visualization modules and utilities for the new database system to determine if the design meets CARB’s business requirements. CARB anticipates the use of automated test tools for testing. Track, document, and report all defects discovered and resolved defects during testing (include test reports in weekly status updates). Fix all reported defects; redeliver and retest all defect repairs.


Phase Deliverable

Data Processing Methods Documentation

Real time Data Streaming Methods Source Code and/or Software

Regulatory Data Upload Source Code and/or Software

Backfill Data Source Code and/or Software

Discrete Laboratory Data Upload Source Code and/or Software

Data Quality Assurance Infrastructure Source Code and/or Software

Data Aggregation Source Code and/or Software

Data Processing Testing Results

Web Page Documention

Operational UserFriendly Web Page

Operational Data Query Tool

Web Page Testing Results

Visualization Tools Documentation

Data Interpretation Documentation

Operational WebBased Data Visualization Tools

Operational WebBased Integration with Other Databases

Visualization Testing Results

Phase

Phase 4. Final Testing, Stabilization, and TransitionDescription

This phase covers the testing of the complete community monitoring air quality system, including database and web portal and system administration. Phase 4 also includes deployment of the system along with training and knowledge transfer. Phase 4 Tasks may include but are not limited to the following:

1. Final System Integration – Complete remaining integration of individual system components to ensure that the entire CAQM system (backend and frontend) meet CARB’s business and technical requirements. Ensure seamless communication between all components between the backend database and frontend web pages.

2. Develop System Monitoring Method – Work with AQPSD and MLD staff to design and develop a method or tool that CARB staff can utilize to oversee the CAQM system. The method or tool may include but is not limited to the following items: information regarding whether or not data streams are functioning properly and data is being received, system emails or notifications for anomalously high or low data, a map displaying which sites/monitors are down or no long transmitting data, and any other tools deemed useful in diagnosing and troubleshooting system errors.

3. Data Archival and System Maintenance – Recommend tools and protocols for data archival and system backup and recovery. Work with AQPSD staff to develop automated programs that archive the CAQM database on daily, weekly, monthly, and annual bases. Consult with OIS staff to determine protocols for periodic full (frontend and backend) system backup and recovery procedures. Create a log table within the database and automated routines that write to the log table that record any jobs run, error codes, reasons for failure, or run completed date/time stamp.

4. Complete System Testing – Perform final functional, integration, performance, and security testing of all system modules and utilities for the new CAQM system to determine if the design meets CARB’s business requirements. CARB anticipates the use of automated test tools for testing. Track, document, and report all defects discovered and resolved defects during testing (include test reports in weekly status updates). Fix all reported defects; redeliver and retest all defect repairs.

5. Source Code and/or Solution Components Transfer – Deliver all final source code or other solution components iteratively throughout the development process via a Statehosted source code version control system or other pathways agreed upon by CARB.

6. Deployment into Production – Upon completion of testing, work with CARB systems operation staff to successfully deploy the initial delivered solution into CARB’s production environment. Provide written build and runtime instructions for each logical server and system process.

7. Full Documentation – Develop and deliver an asbuilt system design documentation that reflects the high level architecture and design of the production system, including hardware performance specifications, software components, user roles, functional processing overview, web application site map, system data flows and interfaces, and entity data model.

8. User Manuals and other Documentation – Develop and deliver programming, operation, and any other requested user manuals and/or materials. Work with CARB staff to determine the most effective ways to distribute as well as develop important information to public including caveats with monitoring methods, sensors, data aggregation techniques, uncertainty, bias, and or other data errors, in addition to information about how the data are collected, transmitted, processed, and stored in the CAQM database.

9. Knowledge Transfer and Training – Provide detailed document walkthroughs of deliverables from Tasks 8.1 and 8.2 with CARB staff. Documentation and walkthroughs must include adequate material, detail, and question and answer periods sufficient to ensure knowledge transfer from Contractor to CARB staff.

Work with CARB software maintenance and operations staff to provide any additional knowledge transfer and/or training of the detailed system design and code, and transition maintenance of the system to the State.

10. Maintenance Support – Provide solution maintenance support, including response to production incidents, problem analysis, defect repairs, data fixes, application changes, system testing, user acceptance testing support, and application builds associated with each production release. All system changes must be delivered via established configuration management processes and procedures. Fix all user reported defects; redeliver and retest all defect repairs.


Phase Deliverable

Completely Operational CAQM System (with integrated frontend and backend)

Operational Method for Administrators to Monitor CAQM System

Operational Data Archival and System Maintenance Procedures

Complete System Testing Results

Complete System Source Code and/or Software

Fully Deployed Production CAQM System

Complete CAQM System Documentation

User Manuals for Web Portal

Knowledge Transfer and Training Materials

Manintenance Support

2.11.6 High Level Proposed Project ScheduleProject Planning Start Date: 9/1/2017 Project Start Date: 7/2/2018

Project Planning End Date: 6/29/2018 Project End Date: 6/29/2019

Activity Name Start Date End DateStage 3 Solution Development 11/1/2017 1/31/2018

Solicitation Development 2/1/2018 3/1/2018

Solicitation Package Review 3/2/2018 4/13/2018

Solicitation Release 4/16/2018 4/30/2018

Solicitation Award 5/1/2018 5/18/2018

Stage 4 Project Readiness and Approval 5/21/2018 6/15/2018

Requirements 6/15/2018 7/13/2018

Design 7/16/2018 9/14/2018

Development 9/17/2018 4/26/2018

Testing 4/29/2019 5/30/2019

Training 5/31/2019 6/6/2019

Deployment 6/7/2019 6/21/2019

Go Live 6/24/2019 7/1/2019

Maintenance and Operations 5/30/2019 12/30/2021

2.11.7 Cost SummaryTotal Proposed Planning Cost: $615,102

Total Proposed Project Cost: $1,663,520

Average Proposed Operations Cost: $244,676

2.12 Staffing Plan2.12.1 AdministrativeThe CARB Administrative Section will support the project by supplying the following positions during the duration of the project.

Position Role Phase PY CommentsStaff Services Analyst

Contract Analyst

PAL (17/18)Project (18/19)M&O (19/20)

0.0670.050.067

Very experienced leading contract management and administration process. Provide guidance and assistance for CAQM's project contract administration process. Existing workload will be allocated to new staff.

Staff Services Analyst

Procurement PALProjectM&O

0.0500

Wellexperienced on procurement process. Provide guidance and assistance procurement solication process for CAQM's project approval lifecycle process. Existing workload will be allocated to new staff.

2.12.2 Business ProgramCARB will meet increasing staff resource needs for this project by allocating the additional workload to current SME's along with several new positions which have already been approved for FY201718. Previous workload for existing staff assigned to this project will be absorbed by additional positions approved for FY201718 which are not assigned to this project.

Position Role Phase PY CommentsAssistant Division Chief

Project Sponsor

PALProjectM&O

0.0750.050

Very experienced and capable sponsor with years of experience in similar efforts. Visibly supports the project at the executive level. Provides oversight and

direction to the Project Director on strategic issues that affect the project. Resolves issues the Project Director is unable to resolve. Will add to current workload.

Air Resources Sup II

Project Director/Business Lead

PALProjectM&O

0.150.10.0

Years of experience leading similar efforts. Leads the Project. Directs the planning and execution of all project activities, team, and resources. Approves all project requirements, project schedule and cost changes. Accepts all project deliverables. Approves invoices. Will add to current workload.

Air Resources Sup II

Organization Change Management

PALProjectM&O

0.10

Has overseen OCM for several implementations of new processes, technical systems, and scientific solutions. Will add to current workload.

Air Resources Sup I

Business Lead PALProjectM&O

0.10.10

Wellexperienced in leading similar efforts. Subject matter expert on criteria and emission inventory data and legacy databases. Lead the team and their associated project activities. Will add to current workload.

Staff Air Pollution Specialist

SME PALProjectM&O

100

It is anticipated that the current workload assigned to a Staff Air Pollution Specialist SME will be reduced during PAL, and distribute its current workload to existing and new hires Staff Air Pollution Specialist colleagues. This SME will be replaced by a new hire ARE during project and M&O.

Air Resources Engineer

SME PALProjectM&O

0.511

New staff to be hired to support the database development project, and ongoing maintenance & future operations as a SME.

2.12.3 Information Technology (IT)

The CARB OIS Division will support the project by supplying the following positions during the duration of the project.

Position Role Phase PY CommentsSenior Information Systems Analyst

Project Manager

PALProjectM&O

0.20.250.1

The OIS Project Management Office has a team of project managers with extensive experience in projects of a similar nature and scale. The .2 PY redirected during the PAL is additional workload. The OIS PM will be supplemented by .55 PY contracted PM during the project timeframe.

Senior Information Systems

Business Analyst

PALProjectM&O

0.21.050

The OIS Project Management Office has a Business Analyst resource with extensive experience in projects of a similar nature and scale. the .2 PY redirected

Analyst during the PAL is additional workload. The OIS BA will be suplemented by 1.05 PY contracted BA for the project timeframe.

Senior Information Systems Analyst

Contract Analyst

PALProjectM&O

0.2580.050

OIS has an IT Acquisitions team of resources dedicated to all IT acquisition issues for the California Air Resources Board (CARB). The team has vast experience and includes staff recognized as experts. The IT Operations and Support Branch Chief, the IT Operations Section Supervisor and the Senior IT Acquisitions Specialist have extensive IT acquisition experience. Combined, they have performed in roles ranging from contract managers, contract analysts, and all have experience in managing an IT acquisitions team. In addition to this highly experienced and seasoned executivelevel team, OIS also has multiple IT acquisition specialists (analysts) with novice experience up to semiexpert level, all focused and dedicated to the unique requirements of State IT acquisitions. The IT Acquisitions team also has a dedicated IT budget analyst to monitor all of ARB’s consolidated IT budget and IT projects.


ProcurementAnalyst

PALProjectM&O

0.14200

OIS has an IT Acquisitions team of resources dedicated to all IT acquisition issues for the California Air Resources Board (CARB). The team has vast experience and includes staff recognized as experts. The IT Operations and Support Branch Chief, the IT Operations Section Supervisor and the Senior IT Acquisitions Specialist have extensive IT acquisition experience. Combined, they have performed in roles ranging from contract managers, contract analysts, and all have experience in managing an IT acquisitions team. In addition to this highly experienced and seasoned executivelevel team, OIS also has multiple IT acquisition specialists (analysts) with novice experience up to semiexpert level, all focused and dedicated to the unique requirements of State IT acquisitions. The IT Acquisitions team also has a dedicated IT budget analyst to monitor all of ARB’s consolidated IT budget and IT projects.

System Software Specialist III

Technical SME

PALProjectM&O

0.20.20.1

OIS technical section has an extensive experience in technical projects of similar nature and scale. The current workload will be added as the technical SME. Part of the normal duties of the existing System Software Specialist (technical SME) is administering the legacy databases.

Staff ProgrammerAnalyst

Technical SME

PALProjectM&O

011

New staff to be hired to support project and ongoing maintenances and operations.

Manager IV Contract Manager

PALProject

0.050.05

OIS Management has extensive experience managing contracts of similar nature and scope. It is anticipated

M&O 0 that the current contract management workload assigned to the Manager IV position will be reduced due to the expiration of current projects, freeing up sufficient resources to man this project.

Manager IV Project Oversight

PALProjectM&O

00.10

OIS Management has extensive experience overseeing projects of similar nature and scope. It is anticipated that the current contract management workload assigned to the Manager IV position will be reduced due to the expiration of current projects, freeing up sufficient resources to man this project.

2.12.4 Testing

The CARB Air Quality Planning and Science Division and Office of Information Technology Staff will work in conjunction with contracted staff to support testing by supplying the following positions during the duration of the project. These internal PYs will be augmented with .5 PY contracted BA.

Position Role Phase PY CommentsSystem Software Specialist II

Security PALProjectM&O

00.20

Security testing, part of normal duties.


SME/Testing PALProjectM&O

011

New staff to be hired to support the database development project, and ongoing maintenance & future operations.


SME/Testing PALProjectM&O

00.80

It is anticipated that the current workload assigned to Air Pollution Specialist SME's will be allocated to the new hires of Air Resources Engineer, freeing up sufficient time to take on tasks and responsibilities for this project.

2.12.5 Data Conversion/Migration

Data management activities will include data cleanup, formatting, and testing. The CARB Air Quality Planning and Science Division and Office of Information Technology Staff will work in conjunction with contracted staff to support data conversion activities by supplying the following positions during the duration of the project.

Position Role Phase PY CommentsStaff Programmer Analyst

Technical SME PALProjectM&O

010

New staff to be hired to support project and ongoing maintenances and operations.


Business Analyst/Migration

PALProjectM&O

0.20

The OIS Project Management Office has a Business Analyst resource with extensive experience in projects of a similar nature and scale. It is anticipated that the current workload assigned to the BA will be reduced due to the expiration of current projects, freeing up sufficient time to work this project.


SME/Migration PALProjectM&O

00.10

It is anticipated that the current workload assigned to a Staff Air Pollution Specialist SME will be reduced, and distribute its current workload to existing and new hires Staff Air Pollution Specialist colleagues.


SME/Migration PALProjectM&O

00.80

New staff to be hired to support the database development project, and ongoing maintenance & future operations.

2.12.6 Training and Organizational Change Management

The CARB Air Quality Planning and Science Division and Office of Information Technology Staff will work in conjunction with contracted staff to support OCM by supplying the following positions during the duration of the project. Since this is primarily a technology upgrade business interruption is expected to be minimal.

Position Role Phase PY CommentsAir Resources Sup II

OCM PALProjectM&O

0.20

Years of experience leading similar efforts. Leads the Project. Directs the planning and execution of all project activities, team, and resources. Approves all project requirements, project schedule and cost changes. Accepts all project deliverables. Approves invoices. Will add to current workload.

2.12.7 Resource Capacity/Skills/Knowledge for Stage 3 Solution DevelopmentCARB has an IT Acquisitions team with vast knowledge of the IT procurement policies and procedures. The acquisition team has well over 15 + years of IT Acquisitions experience. Experience ranges from conducting solicitations, evaluations of responses, to drafting the purchase order and/or contracts agreements.

The IT Acquisitions team is knowledgeable and well versed with the various procurement methods (i.e., RFP, IFB’s, IT RFQs, SB/DVBE Option, Informal/Formal Solicitations, Leverage Agreements, Fair & Reasonable, and NonCompetitively Bids). The IT Acquisitions processes contracts and procurements from small dollar amounts to million dollar agreements. Having a wide range of expertise in the various areas of procurements and contracts staff are experienced with the STPD Streamlined Procurement Template.

CARB’s procurement office is familiar with the protest types and the use of the PCC 6611.

2.12.8 Project Management2.12.8.1 Project Management Risk AssessmentProject Management Risk Score: 1.4

Attach file:

3900_CARB_CAQM_ProjectManagementRiskAssessment.xlsxMicrosoft Excel Worksheet26.4 KB

2.12.8.2 Project Management PlanningAre the following project management plans or project artifacts complete, approved by the designated Agency/state entity authority, and available for Department of Technology review?Project Charter No Currently Under Development

Scope Management Plan No Currently Under Development

Risk Management Plan No Currently Under Development

Issue and Action Item Management Plan No Currently Under Development

Communication Management Plan No Currently Under Development

Schedule Management Plan No Currently Under Development

Human Resource Management Plan No Currently Under Development

Staff Management Plan No Currently Under Development

Stakeholder Management Plan No Currently Under Development

Governance Plan No Currently Under Development

2.12.9 Organization Charts

3900_CARB_CAQM_Org_Chart_Project.pdfAdobe Acrobat Document79.4 KB

3900_CARB_CAQM_Org_Chart_Procurement.pdfAdobe Acrobat Document100 KB

3900_CARB_CAQM_Org_Chart_IT.pdfAdobe Acrobat Document373 KB

3900_CARB_CAQM_Org_Chart_AQPSD_Program.pdfAdobe Acrobat Document422 KB

3900_CARB_CAQM_Org_Chart_Agency .pdfAdobe Acrobat Document5.21 MB

2.13 Data Conversion/MigrationIdentify the status of each of the following data conversion/migration activities:

Data Conversion/Migration Planning Not ApplicableData Conversion/Migration Requirements Not ApplicableCurrent Environment Analysis In ProgressData Profiling In ProgressData Quality Assessment In ProgressData Quality Business Rules In ProgressData Dictionaries In ProgressData Cleansing and Correction In Progress

This proposal is for a completely new system so it is anticipated that data conversion/migration issues will be minimal.

File Attachment

2.14 Financial Analysis Worksheets

3900_CARB_CAQM_FAW.xlsxMicrosoft Excel Worksheet3.18 MB

Department of Technology Use OnlyPreliminary Assessment – Department of Technology Use OnlyOriginal “New Submission” Date 1/9/2018

Form Received Date 2/23/2018

Form Accepted Date 2/23/2018

Form Status Completed

Form Status Date 4/11/2018

Main Form Department of Technology Use OnlyOriginal "New Submission" Date 1/9/2018

Form Received Date 2/23/2018

Form Accepted Date 2/23/2018

Form Status CompletedForm Status Date 4/11/2018

Form Disposition ApprovedForm Disposition Date 4/11/2018

Documents

Stage 2 Preliminary Assessment - California