﻿ RCFS packages = ['numpy']

# Exercise 8Exploration with ergodic control

In this exercise, we will use ergodic control as an exploration mechanism for a point mass agent.

The goal is to analyze how ergodic control can be used to find a small object with an unknown location but with a prior knowledge on the possible location of this object, in the form of a distribution. The location of the object is made visible to you, but is unknown to the robot. The time took by ergodic control to find this object is shown below the animation.

Ergodic control uses a prior information on the location of the object in the form of a probability distribution. In this exercise, a mixture of Gaussians is used as distribution (represented as pink ellipsoids to represent the contours of two standard deviations).

Ergodic control computes control commands that will generate a movement so that the agent will spend time in the different location of the workspace in proportion to the given target spatial distribution. Namely, if we would discretize the workspace and count the number of times the agent passed in each cell of this grid, the controller will ensure that over time, the resulting histogram will match the desired spatial distribution (optimal coverage).

• Change the Gaussian parameters `param.nbGaussian`, `param.Mu`, `param.Sigma` and the initial position of the agent `param.x0` (point in black in the animation). Observe the resulting search behavior.
• Set the target position `param.target` to be a sample from the mixture of Gaussians (with equal mixing coefficients), the means `param.Mu` and the covariance matrices `param.Sigma`.
Hint: to sample from a mixture of Gaussians, you can first uniformly sample between 1 and `param.nbGaussian` to first choose the Gaussian component to sample from. You can then draw a random sample from this Gaussian distibution.

param.x0 = np.array([.2, .3]) param.nbGaussian = 2 param.Mu = np.ones((param.nbVar,param.nbGaussian)) * .5 # Implement here param.Sigma = np.zeros((param.nbVar,param.nbVar,param.nbGaussian)) # Implement here # Sampling from GMM to define the target param.target = np.array([.5, .5]) # implement here update_ergodic_control()
param.x0 = np.array([.2, .3]) param.nbGaussian = 2 param.Mu = np.zeros((param.nbVar,param.nbGaussian)) param.Sigma = np.zeros((param.nbVar,param.nbVar,param.nbGaussian)) for i in range(param.nbGaussian): param.Mu[:,i] = np.random.uniform(0.1,0.9,param.nbVar) sigma_v = np.random.uniform(-1.0, 1.0, param.nbVar) sigma_scale = np.random.uniform(0, 0.1, 1) sigma_regularization = np.random.uniform(0, 0.01, 1) sigma_v = sigma_v / np.linalg.norm(sigma_v) param.Sigma[:,:,i] = np.outer(sigma_v,sigma_v) * sigma_scale + sigma_regularization # Sampling from GMM to define the target gaussian_id = np.random.choice(np.arange(0,param.nbGaussian)) param.target = np.random.multivariate_normal(param.Mu[:,gaussian_id],param.Sigma[:,:,gaussian_id]) param.target = np.clip(param.target, 0.01, 0.99) # Target within [0,1] update_ergodic_control()