This thesis contributes to algorithmic approaches for the motion generation problem for
mobile manipulators. This problem is unsolved in unstructured environments, where the
robot does not have access to precise models but must infer the state of the world with its
sensors. The challenges for motion generation in these problems arise from the uncertainty
prevalent in real world sensors, the different modalities that need to be considered, and real
time constraints. Our approach in this thesis is to combine local feedback control with global
planning under uncertainty to solve three different applications in manipulation.
Proteins are involved in almost all functions in our cells due to their ability to combine
conformational motion with chemical specificity. Hence, information about the motions of a
protein provides insights into its function. Proteins move on a rugged energy landscape with
many local minima, which is imposed on their high-dimensional conformational space. Exhaustive
sampling of this space exceeds the available computational resources for all but the smallest
proteins. Computational approaches thus have to simplify the potential energy function and/or
resolution of the model using information about what is relevant and what can be ignored. The
accuracy of the approximation depends on the accuracy of the used information. Information
that is specific to the problem domain, i.e. protein motion in our case, usually results in better
models.
In this thesis, I propose a novel elastic network model of learned maintained contacts, lmcENM.
It expands the range of motions that can be captured by such simplified models by leveraging
novel information about a protein’s structure. This improves the general applicability of elastic
network models.
Grasping is a crucial skill for any autonomous system that needs to alter the physical world. The complexity of robot grasping stems from the fact that any solution comprises various components: Hand design, control, perception, and planning all affect the success of a grasp. Apart from picking solutions in well-defined industrial scenarios, general grasping in unstructured environment is still an open problem.
In this thesis, we exploit two general properties to devise grasp planning algorithms: the compliance of robot hands and the stiffness of the environment that surrounds an object. The investigations and planning algorithms show that exploiting compliance in hands and stiffness in the environment leads to improved grasp performance.
Intelligent robots must be able to learn; they must be able to adapt their behavior based on experience. But generalization from past experience is only possible based on assumptions or prior knowledge (priors for short) about how the world works.
I study the role of these priors for learning perception. Although priors play a central role in machine learning, they are often hidden in the details of learning algorithms. By making these priors explicit, we can see that currently used priors describe the world from the perspective of a passive disinterested observer. Such generic AI priors are useful because they apply to perception scenarios where there is no robot, such as image classification. These priors are still useful for learning robotic perception, but they miss an important aspect of the problem: the robot.
In this thesis we study robot perception to support a specific type of manipulation task in unstructured environments, the mechanical manipulation of kinematic degrees of freedom. We propose a general approach for interactive perception and instantiations of this approach into perceptual systems to build kinematic, geometric and dynamic models of articulated objects.
Reinforcement learning is a computational framework that enables machines to learn from trial-and-error interaction with the environment. In recent years, reinforcement learning has been successfully applied to a wide variety of problem domains, including robotics. However, the success of the reinforcement learning applications in robotics relies on a variety of assumptions, such as the availability of large amounts of training data, highly accurate models of the robot and the environment as well as prior knowledge about the task.
Raphael Deimel's thesis reconsiders hand design from the perspective of providing first and foremost robust and reliable grasping, instead of precise control of posture and simple mechanical modelabilty. This results in a fundamentally different manipulator hardware, so called soft hands, that are made out of rubber and fibers which make them highly adaptable. His thesis covers not only hand designs, but also provides an elaborate collection of methods to design, simulate and rapidly prototype soft robots, referred to as the "PneuFlex toolkit".
Three-dimensional protein structures are an invaluable stepping stone towards the understanding of cellular processes. Not surprisingly, state-of-the-art structure prediction methods heavily rely on information. This thesis aims to leverage new information sources: Physicochemical information encoded in predicted structure models and experimental data from high-density cross-linking / mass spectrometry (CLMS) experiments. We demonstrate that these information sources allow improved structure prediction and the reconstruction of human serum albumin domain structures from experimental data collected in its native environment, human blood serum.
The key features of this system are a high degree of immersion into the computer generated virtual environment and a large working volume. The high degree of immersion will be achieved by multimodal human-exoskeleton interaction based on haptic effects, audio and three- dimensional visualization. The large working volume will be achieved by a lightweight wearable construction that can be carried on the back of the user.
Computationally efficient motion planning mus avoid exhaustive exploration of high-dimensional configuration spaces by leveraging the structure present in real-world planning problems. We argue that this can be accomplished most effectively by carefully balancing exploration and exploitation.
Exploration seeks to understand configuration space, irrespective of the planning problem, and exploitation acts to solve the problem, given the available information obtained by exploration. We present an exploring/exploiting tree (EET) planner that balances its exploration and exploitation behavior.
The planner acquires workspace information and subsequently uses this information for exploitation in configuration space. If exploitation fails in difficult regions the planner gradually shifts to its behavior towards exploration.
This thesis develops robotic skills for manipulating novel articulated objects. The degrees of freedom of an articulated object describe the relationship among its rigid bodies, and are often relevant to the object's intended function. Examples of everyday articulated objects include scissors, pliers, doors, door handles, books, and drawers. Autonomous manipulation of articulated objects is therefore a prerequisite for many robotic applications in our everyday environments.
The most significant impediment for protein structure prediction is the inadequacy of conformation space search. Conformation space is too large and the energy landscape too rugged for existing search methods to consistently find near-optimal minima.
Robots already impact the way we understand our world and live our lives. However, their impact and use is limited by the skills they possess. Currently deployed autonomous robots lack the manipulation skills possessed by humans. To achieve general autonomy and applicability in the real world, robots must possess such skills.
Humans are experts in operating anthropomorphic hands based on their intuition. However, transfer of intuitive manipulation strategies requires tools that can capture relevant aspects that govern human behaviors.
More specifically, we want to transfer robust human in-hand manipulation strategies to the anthropomorphic RBO Hand 3. At the moment, this hand’s actuators are operated using a mixing board: a tedious and unintuitive process. In this work, we design a general vision-based teleoperation system that lets users operate the robot hand by mirroring intuitively produced movements from their own hand.
The goal of this thesis is to mitigate some of the issues the current air-mass control strategy of the soft-pneumatic-actuators introduces. So far the controller is derived in a data-driven way and does not rely on accurate sensor-based air-mass estimates. Therefore, when running the controller for a longer time duration, the internal air-mass estimates computed by the model are subject to drift. This results in not being able to control the actuators precisely anymore. In the recent past new air-flow-sensors have been developed that might be feasible for our requirements.
more to: Analysis of Soft Finger Pulp Design on Grasping and Manipulation
Robots should be able to examine and understand previously unseen kinematic structures, such as the furniture and devices in a kitchen. What are good strategies for a robot to explore its environment? Could we learn something from biological behavior to improve this aspect of our robots? In a collaboration with colleagues from Vienna, we try to understand how Cockatoos can learn to solve multi-step kinematic puzzles by building models of the birds' behavior.
more to: Modelling and Understanding Cockatoo's Mechanical Problem Solving Behavior
When we build soft robotic grippers, we draw inspiration from the compliance and softness of the human hand. But what makes the human hand one of the most powerful tools is it's tactile sense.
Since the sensors that are established in rigid robotics an not applicable to soft actuators, we have to take a look at new materials and new fabrication methods.
In this thesis, we want to present one possible way to introduce tactile sensing to a soft actuator without restricting its dexterous behavior, using the RBO Hand 3 actuator as an example.
more to: A Fabric Based Tactile Sensor for Soft Robotics
Robot hands are one of the most important but also most complex parts of a robot system. In the field of soft robotics, one goal is to design robot hands that resemble the human hand and can adapt their capabilities. Especially important is the ability to manipulate objects in the hand, the so-called in-hand manipulation. But to carry out in-hand manipulation complex movements are required. In order to be able to execute these movements, we would like to teleoperate the RBO-Hand 3 with a data glove. The aim of this thesis is to remotely control a soft pneumatically operated robot hand developed by the department with the help of a data glove in order to be able to carry out in-hand manipulations and to perform experiments on in-hand manipulation with the RBO-Hand 3.
more to: Teleoperation of a Soft Robotic Hand with a Dataglove
Sensors are an important part of any robot control system. While soft pneumatic actuators can't use most sensors from rigid robotics, they exhibit properties that make new sensing modalities possible. The air inside the air chamber of a pneumatic actuator conducts sound and this sound carries information about where it originated and which path it traveled.
The RBO Hand 2 is a highly compliant soft robotic hand. Its actuators passively adapt their shape to different objects and the environment. Even though the control of the pneumatic hand is relatively simple, it is capable of complex in-hand manipulation.
The recent addition of liquid metal strain sensors has created the opportunity to obtain better feedback about the current state of the hand.
In this thesis, we pick the task of "wiggling" a pen between the fingers and show that data from the strain sensors can be used to classify the position of the pen within the hand. Using a open-loop correction movement, the pen position can be adjusted before it drops from the hand.
We also showed the possibility of closed-loop control using the strain sensor measurements as the current state and mass-flow commands of the pneumatic actuators as actions.
more to: Learning a Repetitive In-Hand Manipulation Task on the Sensorized RBO Hand 2 Using Policy Search
Learning an internal state based on the observation is an important task in robotics. The sensor inputs are mostly high dimensional and only a small subspace is important for the robot. Previous work presented an unsupervised method training a neural net with a loss composed of robotic priors which has been effective in a markovian observation space. In this thesis, we try to extend this method to work on non markovian observation spaces, and train a recurrent network which should transfer the non markovian observations into a internal state space which fulfill the markov property. For this, we adapt the robotic prios to the new task and evaluate our method in a new experimental setting. As a goal, the robot should be able to solve a simple navigation task using only the learned state representation.
Classical robotic grasping approaches employ static behaviors: First the hand is maneuvered to the object, then the fingers are closed, and finally the hand is retracted from the scene with the grasped object. On the other hand, humans execute wrist movements concurrently with the fingers closure. They also demonstrate higher performance in terms of stability. I therefore hypothesize that this coordination of wrist and hand would enable robust robotic grasping.To evaluate this hypothesis, I conducted an experiment with seven human subjects grasping a set of seven objects using a robotic hand. The subjects guided the robotic hand with a handle and closed the fingers at will. I used a compliant robotic arm to limit the subjects’ control of wrist movement to different extents in five experimental conditions.
Many objects in the real world are articulated objects, e.g. office drawers or doors. These are objects made up of rigid bodies connected by joints. Robots are therefore often confronted with these articulated objects in the real world and should be able to successfully interact with them.
In this thesis, we present an approach for retrieving templates that is inde-
pendent of sequence similarity. In order to do this, we combine ab initio with
comparative modeling by looking for similarities in ab initio decoys to identify good
templates.
Most models in contact dynamics show some unrealistic behavior due to assumptions that were made for the sake of computational convenience. Unfortunately there is a lack of experimental work to validate these assumptions and to evaluate how realistic these contact modeling approaches are, which is the purpose of this thesis.
In evolutionary computation, a goal-based objective function is typically unable to include the local challenges on the way towards its fulfillment and tends to cause the search to converge prematurely. Therefore, this work proposes to use objectives that are defined by different aspects of an individuals interaction with the environment and a selection procedure able to reallocate search efforts in order to avoid convergence. The objectives, curiosity, novelty and evolvability, differ in the time-scale they operate over and the amount of information they include about the problem structure. The common theme of these objectives is their tendency to increase the diversity of behaviors, which is assumed can act as general-purpose utility value.
more to: Entropy as an Organizing Principle for Selection in Evolutionary Robotics
In this thesis we present a new contact predictor that combines evolutionary, sequence-based and physicochemical information. The contact predictor uses a new and refined feature set with drastically reduced dimensionality.
Interactive Perception exploits the robot capabilities to interact with the environment to reveal hidden properties, like the kinematic structures of articulated objects. However, when the robot faces a new environment, it needs to decide on how to interact to maximize the information gain based on sensor data, and use compliant controllers that allow the articulation to guide the motion...
The ability to selectively stiffen otherwise compliant soft actuators increases their versatility and dexterity. The thesis investigates granular jamming and layer jamming as two possible methods to achieve stiffening with PneuFlex actuators, a type of soft continuum actuator. It details five designs of jamming compartments that can be attached to an actuator. The strength of the most effective prototype based on layer jamming is also validated in the context of pushing buttons.
We present an incremental method for motion generation in environments with unpredictably moving and initially unknown obstacles. The key to the method is its incremental nature: it locally augments and adapts global motion plans in response to changes in the environment, even if they significantly change the connectivity of the world.
In this thesis we present a motion generation approach that shifts the boundary between planning and control methods to generate task-consistent motion under uncertainty.
Sequence alignment methods are frequently used in protein structure prediction to identify homologous protein structures. The existing methods make local and global alignments between sequentially contiguous protein sequences. However, in our ongoing protein structure prediction research, we have a unique sequence to sequence alignment problem. The potential sequence alignments need to be made between a target sequence and the sets of sequence fragments, where the sequence fragments may not be sequentially contiguous.
The introduction of protein fragment libraries in the mid-1990s meant a huge leap forward for protein structure prediction. With them, one was able to combine the best matching parts from a “scrapyard” of protein parts instead of focusing on just one template protein.
Up until today, commonly used fragment libraries only contain relatively small, independent fragments. Consequently, these libraries can only model the (sequentially) local context, but can’t model structurally conserved regions that are sequentially discontiguous. We therefore developed a library of so called ”building blocks”. A building block is a set of structurally contiguous, sequentially discontiguous fragments found in two or more proteins.
A novel approach for the protein loop closure problem inspired from
robot modeling is developed using the kinematic chain representation of
the loop chain and a motion planning technique.
Haptic devices enhance the range of multi-modal interaction in virtual reality environments. With the wearable haptic device, developed at the RBO Lab, this interaction is not limited to a small workspace any longer. The wider range of motion allows for new application scenarios.
The goal of this thesis is to mitigate some of the issues the current air-mass control strategy of the soft-pneumatic-actuators introduces. So far the controller is derived in a data-driven way and does not rely on accurate sensor-based air-mass estimates. Therefore, when running the controller for a longer time duration, the internal air-mass estimates computed by the model are subject to drift. This results in not being able to control the actuators precisely anymore. In the recent past new air-flow-sensors have been developed that might be feasible for our requirements.
more to: Air-Mass Control of Soft Pneumatic Actuators with Air-Flow-Sensors
The thesis builds on the work from Ines Putz, applying deep neural networks from the contact prediction field to breaking contact prediction during protein motion. Elastic Network Models are useful in determining the coarse-grained motion of proteins. During structural transitions, certain residue pairs that were spatially close become separated, so called breaking contacts. Incorporating this information improves the prediction of protein motion. In this thesis, we want to apply state of the art machine learning methods to breaking contact prediction.
more to: Convolutional Neural Networks for the prediction of breaking contacts during protein motion
Compliance in soft hands can be both beneficial and detrimental to functionality. Although recent work has shown the benefits of compliance to object and environment geometry, there is little work in identifying and avoiding the negative aspects of compliance while controlling soft hands. However, a planner or a feedback-controller that exploits compliance should avoid the regions of detrimental morphological computation and guide the interactions to the favorable ones.
Luckily recent work in simulation has shown promising results in differentiating between beneficial/detrimental morphological computations. The challenge ahead is to whether these results can be transferred to real systems. Our lab's work in hand sensorization is a possible tool in this path.
For drug design it is essential to know which ligands can reach the active site of a protein. These ligands are potential candidates that inhibit or activate the given protein, and thereby cure a disease. We will show how to use sampling-based motion planning to solve protein-ligand disassembly problems.
Sensing for soft continuum actuators as a necessary technology has emerged recently with the development of so called soft hands, which exploit the high deformability of soft structures and materials. Unfortunately, soft, stretchable sensors capable of withstanding a stretch of 100% are commercially not available. At the same time their tight integration into actuators is required to address the specific challenges of continuously deforming actuators. The thesis evaluates three potential sensor technologies for their suitability in soft hands. The thesis investigates their robustness, ease of use, long term stability and responsiveness with respect to the intended application in soft hands.
To perform fine manipulation tasks without error, it is necessary for the robot to position his fingers with very high accuracy and precision. Using a sophisticated model, a recursive pose estimation by a particle filter based on visual data can provide this, but may fail to provide the necessary speed. My work therefore focuses on estimating the influence of different model parameters to the accuracy and speed of the pose estimation. The most important parameter is the number of particles, which increases the accuracy strongly up to a number of 500. In addition, the computational cost scales linear with the number of particles. Different techniques using the GPU to reduce the time per particle are presented and evaluated.
Predictive state representations (PSRs) are gaining a lot of attention in the robotics community lately because, in theory, they promise a powerful model that might be learned directly from data. But the practical application of PSRs remains a difficult procedure. In this practical guide we aim to ease and encourage practical work with PSRs.
more to: A Practical Guide to Transformed Predictive State Representations
To extend the field of application of robots in unstructured environments it is necessary to develop new techniques of environment perception and interpretation. These methods must give machines the capability to generate sufficient information, which enables them to fulfil their tasks with the aid of their sensors. Therefore it is required to extract local and task dependent invariant structure out of the unstructured environment.
more to: Position-Based Servoing via Probabilistic Part-Based Object Models
Enable a robot to infer the type of joint between moving clusters of 3d features. This can further be used to build a kinematic structure of an observed object.
more to: Three dimensional Joint Detection
