In recent years there has been tremendous progress in the development of Artificial Intelligence (AI), Autonomy, and Augmentation (AAA) technologies, but less success is translating concepts into operationalizing capabilities. As a trusted agent to the DoD and IC, ARLIS applies a human-centered approach to incorporate into operational workflows AAA technologies that are trusted, reliable, and safe.
ARLIS supports operationalizing AAA technologies on multiple fronts: trusted test and evaluation, human-system integration, and foundational AAA research and development (R&D) in support of the first two categories.
- Trusted support work at the systems engineering level has included test and evaluation of integrated systems against curated data and scenarios and evaluate the security and robustness of AAA technologies (e.g., via red teaming via adversarial methods).
- In our mission- and human-centered analysis and evaluation of technology, we work with government sponsors to perform workflow analysis and mission modeling, design and evaluate human-machine teams, and conduct user-centric operational test and evaluation.
- ARLIS is performing foundational AAA R&D in pursuit of operationalization. This includes formal methods and simulation-based verification to support test, evaluation, verification, and validation (TEV&V) for artificial intelligence and autonomy and peripheral nerve stimulation and blended reality displays for prototyping and demonstrating next-generation cognitive augmentation.
The National AI Engineering Initiative: At the request of the Office of the Director of National Intelligence (ODNI) and in partnership with Carnegie Mellon University's Software Engineering Institute, ARLIS is heading up the National AI Engineering Initiative building an R&D roadmap for AI system engineering. This process facilitates research across academia focused on developing tools and methodologies for a robust process, which includes formal methods for AI-based system specification and verification, simulation-based test and evaluation, man-machine teaming, and human-computer interaction.
Putting Human-System Integration to the Test
ARLIS is currently developing a testbed to evaluate AAA technologies at the mission-level for the intelligence community --- to maximize the overall quality of support the tools provide, identify how they might be best employed, and anticipate how they might be harmful. This facility will enable ARLIS to test a wide range of scenarios in technical settings and operational user workflows. As part of this effort, ARLIS is building out a new Human-Computer Interaction Laboratory (HCIL),) within its secure facility, which will accommodate usability and accessibility evaluation for a wide range of AAA technologies, including information visualization, extended reality, and artificial intelligence solutions.
The HCIL will be used for tasks ranging from incremental usability evaluation early in development to final test and evaluation feedback. By ensuring that the technologies that operators and analysts will be using in the future are optimized at the sociotechnical system level, ARLIS testbeds and labs will increase effectiveness for new, emerging AAA technologies and improve capabilities across the IC and DoD.
Physiological Factors of Performance
The field of human performance augmentation is in a moment of rapid expansion, and ARLIS research is at the forefront.
Current Ventures
A Trusted Partner to Enable AI Innovation
The ARLIS team is a trusted test and evaluation (T&E) partner for the government for newly developed AI/ML-accelerator technologies. These technologies include cutting-edge, experimental hardware designed to support hybrid sparse-dense machine learning workflows and large-scale graph analytics, and other algorithms to support a variety of mission-critical tasks.
ARLIS is playing a key role in every step of the T&E life cycle, from problem identification and dataset selection/generation to performance analysis, including scalable, reproducible benchmarking for the purpose of trade-off aware comparative analysis.
AI and VR to Augment Mission Planning
In the past two years, ARLIS worked with a government team to develop a prototype distributed collaboration tool based on virtual reality (VR) technology. The prototype enables physically distributed commanders to plan and rehearse a mission around a virtual "sand table" as if they were physically co-located. The distributed planners collaborate around a virtual representation of the physical terrain, building plans using standard military symbology, and then rehearsing the scenarios through simulation, over-constrained communication channels
The ARLIS team is a trusted evaluation partner on the DARPA's Hierarchical Identify Verify Exploit (HIVE) and Software-Defined Hardware (SDH) programs — both. Both of these programs are part of DARPA's Electronics Resurgence Initiative —, and in this capacity, is developing and running experiments for the AI/ML-accelerator technologies which are being developed under the programs. These technologies include cutting-edge, experimental hardware designed to support hybrid sparse-dense machine learning workflows and large-scale graph analytics, as well as algorithms to support a variety of mission-critical tasks on operational datasets.. ARLIS has played a crucial role in every step of the testing and evaluation life cycle, from problem identification and dataset selection/generation to performance analysis to scalable, reproducible benchmarking for the purpose of trade-off aware comparative analysis.
In 2019 and 2020, ARLIS worked with a team from the Army Futures Command C5ISR Center to develop a prototype distributed collaboration tool based on virtual reality (VR) technology. The prototype enables physically distributed commanders to plan and rehearse a mission around a virtual "sand table" as if they were physically co-located. The distributed planners collaborate around a virtual representation of the physical terrain, building plans using standard military symbology, and then rehearsing the scenarios through simulation, over-constrained communication channels.
|
|
|