OODA loop for learning open-world novelty problems

Open-world learning is a machine learning paradigm that deals with learning from a dynamic and open environment where new categories or classes can be added or removed at any time. In an open-world environment, an intelligent agent is expected to learn the goal independently by interacting with the environment and adapting well to unseen changes. This chapter discusses a proposed RL framework called OODA-RL. It is derived from the four stages of Boyd's OODA loop, popularly used in warfare strategy planning and implementation. O stands for Observe, the second O stands for Orient, D stands for Decide, and A stands for Act. In the “Observe stage,” the Agent receives raw unstructured data as observations from the environment about the surroundings. In the “Decide stage,” the primary role is of a decision-maker, deciding whether the learned state representations are classified as a seen or unseen scenario. Next, the orient stage plays two roles: (i) it creates a continual learning simulated environment based on the raw observations it receives from the observe stage, and (ii) it acts as the expert system in the AL loop between the decide and orient stages. Finally, in the “Act stage,” the Agent takes appropriate action as decided through the OODA-RL loop. Compared to the abstract definition of the classical RL framework, OODA-RL broadens the Agent's internal composition to enable RL researchers to integrate novelty adaptation techniques as an additional feature into both current state-of-the-art and future RL algorithms. In addition, this chapter discusses some of the current works to highlight different approaches to developing open-world learning agents.