Google's DeepMind defeats reigning AlphaGo Champion (2016). Many products of AI research are so common they no longer perceived as AI by the users: ...
UNCLASSIFIED//APPROVED FOR UNCLASSIFIED//APPROVED FOR PUBLIC PUBLICRELEASE RELEASE
AI, Robotics and Cyber: How Much will They Change C2? Dr. Alexander Kott ARL Chief Scientist UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
B3 and C2 Command and Control BOTS (AI)
BITS (Cyber)
2
UNCLASSIFIED
BODIES (Human)
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
AI, Cyber, Humans in a Very Complex World
3
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Everything is Connected
• AI is making the world more intelligent • AI makes the world harder to manage • AI makes the world more vulnerable to cyber • Humans make C2 more vulnerable • Humans complicated things for AI • Humans can add resilience • Cyber thrives on attacking AI • Cyber and humans don’t mix well • Cyber defense will benefit from AI
4
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
AI and Cyber Make C2 Increasingly Complex
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Intelligent Things will be Diverse
Munitions
Sensors Weapons Wearable Devices
Robots
Vehicles
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
They will Perform a Variety of Tasks Sense
Attack
Collect & Process Information Fix
Sustain
Collaborate
Communicate
Defend
with each other & human Soldiers
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Managing and Adapting
Number of nodes for a future Army brigade might be several orders of magnitude greater than in current practice
Million Things per square kilometer is not an unreasonable expectation
Can be advantageous: availability of very large, densely-positioned number of Things, such as sensors
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Human Cognition will be Challenged The human cognition bandwidth will emerge as the most severe constraint. Humans seek infor mation that is: Well-formed Reasonably-sized Essential Highly relevant to their current situation and mission Unless information is useful, it is likely to do more harm than good
Similar concerns apply to all intelligent things
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Human Cognition will be Challenged - 2 Will far exceed advances predicted by Moore’s Law Will far exceed any likely improvements in bandwidth
Volume and complexity of information will be truly unprecedented in their extent
Similarly, trustworthiness and value of information arriving from different things will be highly variable Compression and fusion of data into information would have to be by a factor of 10^15
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Complexity of Intelligent Things: Use it as a Smoke Screen?
Friendly forces will be challenged to find, manage, aggregate information Any one device and its information is vulnerable to cyber or physical capture Use Intelligent Things to disperse friendly information, make any one device useless to the adversary Increase resiliency of C2, confuse and deceive the adversary Kott, A., Swami, A., and West, B., "The Fog of War in Cyberspace," IEEE Computer, November 2016
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
AI and Cyber Make C2 Increasingly Vulnerable
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
They will Have to Deal with the Adversary
Adversarial nature of the environment is the primary concern
Kinetic Directed Energy Electronic Attacks Against its Things Jamming RF Channels Destroying Fiber Channels Depriving Things of their Power Sources Electronic Eavesdropping Deploying Malware
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Humans are Especially Vulnerable Perhaps most importantly, the enemy attacks the cognition of human Soldiers Humans will be “Intelligent Things” that are most susceptible to deceptions Humans’ use of C2 will be handicapped when they are concerned (even if incorrectly) that the information is untrustworthy
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
AI will Fight Cyber Attacks
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Intelligent Cyber Agents
The battle of Cyber domain will continue to grow in significance Offense is stronger than defense Intelligent cyber agents are a possible answer 16
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Intelligent Cyber Agents
Stymie the enemy’s cyber intrusions by believable honeypots and honeynets
Fight back by anomaly detection that can highlight unexpected patterns
Use continuous learning process
Large-scale physical fingerprinting
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Can We Build Defensive Intelligent Agents?
Managing a variety of responses is error-prone Core machine intelligence must reside on-board
ADS is a prototype unified framework for on-board plug-in active defenses ELIDE is learning-based tool for extremely light, on-board intrusion detection
Chang, Raymond J., Richard E. Harang, and Garrett S. Payer; Extremely UNCLASSIFIED Lightweight Intrusion Detection (ELIDe); ARL, 2013
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Resilience will Replace Security
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Definition
Resilience is a property of a system defined by the National Academy of Sciences (NAS) as “the ability to prepare and plan for, absorb, recover from, and more successfully adapt to adverse events”.
Courtesy of: Igor Linkov, M. Bates, A. Ganin, A. Gutfraind, E. Massaro, A. Steen, N. Steen, and M. Wood (ERDC)
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Modeling Help is Needed
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Assessing Impact via Models restoreHost()
Create Alert
malwareDetected()
Between(1,3)h AvailabilityAlert
Restore Functionality
Malicious Activity Discovered ?
Making sense of C2 means relating it to mission impact
Yes Submit Alert
CyCS-deleteTicket() No getNextAlert()
Get Next Alert
Alert Type ?
takeHostOffline()
wipeHost()
5m
Between(1,3)h
5m
Take offline
Wipe and Restore
Put online
WipeAlert
CyCS-deleteTicket() CyCS-createTicket()
Submit Alert InfectedAlert IntegrityAlert ConfidentialityAlert getInfectionSource()
ForensicAlert
None
putHostOnline()
CyCS-deleteTicket()
Create Alert
submitAlert()
getAllInfected()
Between(1,3)h
0m
Between(2,6)h
Between(3,9)h
Trace Attack Source
Issue New Alert
Get Signature
Find other infections
No 0m Targets Available ?
Yes
Issue New Alerts
No alert present
Create Alert getWait()
submitAlert()
Wait to Issue Alert
Issue Alert
Release Resource
Mission is a nexus of a numerous physical assets, information, activities, friendly, enemy…
Start Defender
AMICA explored comprehensive models that cove infrastructure, missions, defenders and attackers Results are insightful but modeling is labor intensive S. Noel, J. Ludwig, P. Jain, D. Johnson, R. Thomas, J. McFarland, B. King, S. Webster and B. Tello, "Analyzing Mission Impacts of Cyber Actions," in Proceedings of the NATO IST-128 Workshop on Cyber Attack Detection, Forensics and Attribution for Assessment of Mission Impact, Istanbul, 2015.
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
AI Must Become Smarter
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Hypes & Successes AI was over-sold many times, but the last 10 years seen impressive achievements Self-driving cars (DARPA Grand Challenge, 2004 no one finished, in 2005 5 teams finished) Urban Challenge (2007 follow-up to previous Grand Challenge, 11 teams competed with 6 finishing, 3 in under 6 hour limit) Spectrum Collaboration Challenge Calls for Contenders (2016) Deep Blue decisively defeats any human chess player (1996 lost, 1997 won after HW upgrade) Watson defeats Jeopardy! Champions (2011) Apple’s Siri (2011), Google Now (2012), Microsoft’s Cortana (2014) AI personal assistants with voice recognition Skype can translate your conversation in real-time (2015) Google’s DeepMind defeats reigning AlphaGo Champion (2016) Many products of AI research are so common they no longer perceived as AI by the users: route planning on Google Maps; Google Translate; facial recognition products; automated customer service; video games; etc. UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces 24
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Real-world C2 is Still Too Hard for AI Highly-dispersed team of human & robot agents accessing highly heterogeneous information sources Dynamic in-flight learning & re-planning at the Speed of the Fight
Learning in new environments with deception from advanced persistent threats
The Army AI and ML problems involve unique challenges: unstructured, unstable, rapidly changing, chaotic, rubble-filled adversarial environments; learning in real-time, under extreme time constraints, with only a few observations that are potentially erroneous, of uncertain accuracy and meaning, or even intentionally misleading and deceptive.
25
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Gaps Vision: Artificially intelligent agents (heterogeneous & distributed) that rapidly learn, adapt, reason & act in contested, austere & congested environments
Gaps AI & ML with small samples, dirty data, high clutter AI & ML with highly heterogeneous data Adversarial AI & ML in contested, deceptive environment Distributed AI & ML with limited communications AI & ML computing with extremely low size, weight, and power, time available (SWaPT) Explainability & programmability for AI & ML AI & ML with integrated quantitative models
26
UNCLASSIFIED
Learning in Complex Data Environments
Resource-constrained AI Processing at the Point-of-Need Generalizable & Predictable AI
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
AI-enabled Capabilities
A Soldier supported by team of agents in complex environment
AI-enabled reasoning to provide possible course of actions
AI-enabled real-time estimates of enemy
UNCLASSIFIED
Tactically sensible decision making based on locally available information
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Humans and AI will Team
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Challenges of Teaming
A key challenge is to enable Intelligent Things and Soldiers to effectively and naturally interact across a broad range of warfighting functions, with trust and transparency, common understanding of shared perceptions, and human-agent dialog and collaboration.
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Intelligent Teams Creating Cohesive, Collaborative Teaming
Providing Agents Greater Capabilities
Accelerated Learning for a Ready Force Artificial Intelligence & Machine Learning MACHINE LEARNING
NATURAL LANGUAGE PROCESSING
PLANNING
Providing Humans Greater Capabilities
UNCLASSIFIED
KNOWLDEGE REASONING
COGNITIVE SYSTEMS
ARTIFICIAL INTELLIGENCE
PERCEPTION
ROBOTICS
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Joint Human Human-Intelligent Agent Decision Making Opportunity:
Plans:
Conceptual & technological advancements in fields such as Human Computation are challenging typical approaches of human system integration
Humans learning from intelligent agents.
Recent dramatic advances in deep learning are changing perspectives on what intelligent agents can do in joint collaboration with humans Enablers: Deductive reasoning using sparse data, hybrid supervised-unsupervised techniques for transfer learning, transfer learning for retraining neural networks, joint learning in unbounded environments
Human observation of intelligent agents can lead to novel understanding of the problem space & novel solution ideas
Intelligent agents can provide assistance with long-term training and decision making
Explainable intelligence underlying efficient integration of cognitive-assist agents.
Reconceive problems to support human cognition Investigate human cognitive capabilities & recast computational problems into domains more amenable to those capabilities
Understandable explanations of “incomprehensible” intelligent agent behaviors
Intelligent Agents Learning from Limited Data.
UNCLASSIFIED
Training deep networks from sparsely-labeled data under time constraints
Learning with statistically mismatched data sparse/adversarial: statistics do not match reality
31 The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Teams Train
AI will be a key technology for building, realistic, intelligent entities in immersive training simulations. These should include realistic sociocultural interactions between trainees and simulated intelligent agents. UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Humans vs. Things in C2
Kott, A. and Alberts, D., “How Do You Command an Army of Intelligent Things?” IEEE Computer, 2017 (to appear)
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Humans vs. Things: Command Things are good at: • • • • •
setting goals and priorities, establishing rules, and setting constraints. assigning roles and responsibilities and defining relationships
Well defined goals and priorities Detect inconsistencies No personal agendas No group think No vague interpretations
Not so much: • • • • • •
Understanding the implied Handle lack of formal, specific Understandable to humans Establishing trust Negotiating goals and constraints Adjusting roles and responsibilities
Kott, A. and Alberts, D., “How Do You Command an Army of Intelligent Things?” IEEE Computer, 2017 (to appear)
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Humans vs. Things: Control
Things are good at: • • •
adjustments to actions, considering changes in the situation and the progress; agility in dynamic environments
Recognition and acceptance of new situation Speed of processing No barriers to overturning past decisions
Not so much: • •
Explaining the recommendations / decision Resolving disagreements
Kott, A. and Alberts, D., “How Do You Command an Army of Intelligent Things?” IEEE Computer, 2017 (to appear)
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
Summary •
C2 will be for and by the distributed society of humans and intelligent things
•
This C2 will be far more fluid and self-adaptive than today’s
•
Proliferation of intelligent things invites predation of malicious cyber agents
•
As well as great increase in overall complexity of C2
•
C2 of this society brings qualitatively new challenges
•
Humans will be both sources of vulnerability and resilience
•
AI both invites cyber-attacks, and enables their defeat
•
Complexity can be exploited for defensive purposes
•
Design of such C2 calls for novel types of modeling and simulation
•
AI for C2 will have to close gaps: adversity, complexity, resource constraints, explainability
•
New forms of human-agent teaming will emerge
•
Humans and Intelligent Things bring complementary strengths to C2
•
Humans will learn to partner with Intelligent Things
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
References
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
References REFERENCES Cho, J.H., Swami, A. and Chen, R., 2011. A survey on trust management for mobile ad hoc networks. IEEE Communications Surveys & Tutorials, 13(4), pp.562-583. Colbert, E., A Kott (eds.), Cyber-security of SCADA and Other Industrial Control Systems, Springer 2016 Ganin, A. A., Massaro, E., Gutfraind, A., Steen, N., Keisler, J. M., Kott, A., & Linkov, I. (2016). Operational resilience: concepts, design and analysis. Scientific reports, 6. Gil, Santiago, Alexander Kott, and Albert-László Barabási. "A genetic epidemiology approach to cyber-security." Scientific reports 4 (2014). Gregory, J.M., Twigg, J.N. and Fink, J.R., 2016, October. Enabling autonomous information-gathering and selfrecovery behaviors in partially-known, communication-constrained environments. In Safety, Security, and Rescue Robotics (SSRR), 2016 IEEE International Symposium on (pp. 92-99). IEEE. Heckman, K., and F. Stech, Cyber Denial, Deception and Counter Deception: A Framework for Supporting Active Cyber Defense. Springer 2015 Kenma, S. Rogers, J. Nieto, C. Sukhatme, G. "A multi-robot coordination strategy for informative adaptive sampling in communications-constrained environments" ICRA 2017 (accepted, to appear) Kott, A. and C. Arnold, Promises and challenges of continuous monitoring and risk scoring. IEEE Security and Privacy, 11(1), 90–93 (2013) Kott, A., Swami, A. and McDaniel, P., 2014. Security outlook: six cyber game changers for the next 15 years. Computer, 47(12), pp.104-106. Kott, A., Swami, A. and West, B.J., 2016. The Fog of War in Cyberspace. Computer, 49(11), pp.84-87. Kott, A., Swami, A. and West, B.J., 2016. The Internet of Battle Things. Computer, 49(12), pp.70-75. Kott, Alexander, David S. Alberts, and Cliff Wang. "Will Cybersecurity Dictate the Outcome of Future Wars?" Computer 48.12 (2015): 98-101.
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces
UNCLASSIFIED//APPROVED FOR PUBLIC RELEASE
References REFERENCES Kott, Alexander, Nikolai Stoianov, Nazife Baykal, Alfred Moller, Reginald Sawilla, Pram Jain, Mona Lange, and Cristian Vidu. "Assessing Mission Impact of Cyberattacks: Report of the NATO IST-128 Workshop." arXiv preprint arXiv:1601.00912 (2016). Kott, Alexander. "Towards fundamental science of cyber security." Network Science and Cybersecurity. Springer New York, 2014. 1-13. arXiv:1512.00407 Lange, M., R. Moeller, G. Lang and F. Kuhr, "Event Prioritization and Correlation based on Pattern Mining Techniques," in 14th International Conference on Machine Learning and Applications, Miami, 2015. Linkov, I., . Eisenberg, K. Plourde, T. Seager, J. Allen, and A. Kott. "Resilience metrics for cyber systems." Environment Systems and Decisions 33, no. 4 (2013): 471-476. Noel, S., J. Ludwig, P. Jain, D. Johnson, R. Thomas, J. McFarland, B. King, S. Webster and B. Tello, "Analyzing Mission Impacts of Cyber Actions," in Proceedings of the NATO IST-128 Workshop on Cyber Attack Detection, Forensics and Attribution for Assessment of Mission Impact, Istanbul, 2015. Rasch, R., Kott, A., & Forbus, K. D. (2003). Incorporating AI into military decision making: an experiment. IEEE Intelligent Systems, 18(4), 18-26. Summers-Stay, D., Teo, C.L., Yang, Y., Fermüller, C. and Aloimonos, Y., 2012, October. Using a minimal action grammar for activity understanding in the real world. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on (pp. 4104-4111). IEEE. Summers-Stay, D., Voss, C. and Cassidy, T., 2016. Using a distributional semantic vector space with a knowledge base for reasoning in uncertain conditions. Biologically Inspired Cognitive Architectures, 16, pp.34-44.
UNCLASSIFIED
The Nation’s Premier Laboratory for Land Forces