Predicting the Force Map of an ERT-Based Tactile Sensor Using Simulation and Deep Networks

2023

Article

al

hi

Electrical resistance tomography (ERT) can be used to create large-scale soft tactile sensors that are flexible and robust. Good performance requires a fast and accurate mapping from the sensor's sequential voltage measurements to the distribution of force across its surface. However, particularly with multiple contacts, this task is challenging for both previously developed approaches: physics-based modeling and end-to-end data-driven learning. Some promising results were recently achieved using sim-to-real transfer learning, but estimating multiple contact locations and accurate contact forces remains difficult because simulations tend to be less accurate with a high number of contact locations and/or high force. This paper introduces a modular hybrid method that combines simulation data synthesized from an electromechanical finite element model with real measurements collected from a new ERT-based tactile sensor. We use about 290,000 simulated and 90,000 real measurements to train two deep neural networks: the first (Transfer-Net) captures the inevitable gap between simulation and reality, and the second (Recon-Net) reconstructs contact forces from voltage measurements. The number of contacts, contact locations, force magnitudes, and contact diameters are evaluated for a manually collected multi-contact dataset of 150 measurements. Our modular pipeline's results outperform predictions by both a physics-based model and end-to-end learning.

@article{Lee23-TASE-Map,
  title = {Predicting the Force Map of an {ERT}-Based Tactile Sensor Using Simulation and Deep Networks},
  author = {Lee, Hyosang and Sun, Huanbo and Park, Hyunkyu and Serhat, Gokhan and Javot, Bernard and Martius, Georg and Kuchenbecker, Katherine J.},
  journal = {IEEE Transactions on Automation Science and Engineering},
  volume = {20},
  number = {1},
  pages = {425--439},
  month = jan,
  year = {2023},
  doi = {10.1109/TASE.2022.3156184},
  month_numeric = {1}
}