Perception and Action Psychology
The last article on visual perception focuses on J.J. Gibson’s ecological approach to perception, which says perception is not passive and it is essential for action in order to interact with the environment. We will see how perceptual input leads us to better navigate the environment through, the concept of optic flow. The latter part of the article covers the concept of affordance; dorsal & ventral pathways of the brain, today we will be talking about the link between perception and action you have seen now that we have studied visual perception in a lot of detail we have seen different aspects of perception the theoretical issues and we have also seen in one of the most recent articles about how visual perception helps us recognize and interact with objects in the environment in the real world we are trying to.
Now go towards a position where we try and say that perception is not really a passive process perception is almost an active process and basically, it changes the way we interact with the environment and that is basically the approach that JJ Gibson took. if you remember that we talked about the ecological approach to perception and that ecological approach to perception was basically about the fact that perception is not a passive process perception is about interacting with the environment and the influence of perception and action is something that is of interest to people who study perception.
Theoretical Background
Now the theoretical background as i said is of JJ Gibson’s ecological approach to perception he believed that perceptions should be studied as people move through the environment and interact with it so his idea was that perception is not really a static thing you are not sitting or standing at somewhere in a stationary manner and then looking and interacting and then looking at the environment around you are more often than not moving around in the environment you’re changing your position with respect to your changing your head position, you’re changing your body position and with that change with that aspect of dynamicity the perception that you are receiving the sensory input that you are receiving is also changing accordingly so that is a brief background with which j.j Gibson’s approach to perception was Stanced and we have studied that in enough detail just to add to whatever we know already or just to remind you I’ll just say that the ecological approach to perception is basically focused itself on moving observers.
So, it is designed in such a way that it is about not stationary observers and how they look at the environment but moving observers and how that movement information influences their perception of the environment this was basically a very important concept in Gibson’s approach to perception was the concept of optic flow if you remember.
What is Optic Flow?
Optic flow is basically the information about our movements in the environment and the aspects in the environment say for example if you remember I talked about the moving train example and how a driver looks out of the train and he sees cows grazing in the field how the movement of the train affects the perception of those cows which are grazing next to the train in the field so those kinds of things about the direction of the movement speed of movement relative distance between the two moving objects these kinds of things figured as very prominent sources of information in Gibson’s approach to perception and that is what we will be building upon in today’s article in trying to make a link between perception and action.
Gradient of Flow
Optic flow basically is supposed to be rather fast near the observer who is moving and it becomes slower as you move further away from this observer so this difference in the intensity of optic flow is referred to as gradient of flow.
Focus of Expansion
This is a very important source of information the other important source of information is basically the focus of expansion if you remember I took an example of a person standing in front of the train and it’s moving in the direction of the train in that sense there will be a small point in much further away from the moving train in the direction that the train is moving and that point will be the point which is called the focus of expansion.
So, it seems that the flow is emanating from that point but that point itself has no optic flow at all so that is a point where there is no optic flow but all the optic flow emanating from that direction at that point is called the focus of expansion.
I have a figure here to show you that the figures from Goldstein’s book of sensation and perception you can see that if you are moving in a particular direction you can see that at that point in the center there is no flow but the flow is actually emanating from that direction so this is what the focus of expansion is here you can see the focus of expansion in the reference to a plane which is landing on an airstrip and you can see again the center point where there is no flow present at all but all the flow is emanating from the sides of those points now another important aspect of the ecological approach to perception is the aspect of invariant information.
What is Invariant Information?
Invariant Information is that information that does not change with respect to the movement of the observer till now we have been talking about how the observer moves around in the environment which causes changes in the perception and that information that changes in perception in terms of optic flow or focus of expansion gives some very useful information to the observer in order to continue his movements or in order to continue his exploration of the environment there is however part of the information that does not really change.
Whether the observer is moving or not and this information is referred to as invariant information as soon as the observer stops moving around the environment there is no flow information there because the flow is contingent on movement the focus of expansion also shifts as soon as the observer changes its direction or it stops moving so these are the two sources of information you will see that the observers use in order to interact successfully with the environment so how does this work if you are moving in the environment I will be creating something called self-produced information?
Self-Produced Information
My movement in the environment will create some perceptual information that will help me continue moving and interacting with the environment if a person is moving along the street say for example imagine yourself driving a car in a particularly busy street while you’re driving the relative distance the direction of your movement and the speed with which you’re moving will change a lot of information about say for example if there is a food stall somewhere nearby to your left or to your right if you the kind of turn that you’re going to approach or with respect your the relative size of the people that are in a particular distance those kinds of information do change when you’re moving.
So, what you’re doing here is you’re actually producing some information some new information that is being used by the perceptual system but this new information is actually being produced by your movement itself and as you change the direction of the movement speed of movement say for example whether you are stopping altogether at all these kind of things will basically be useful for you to plan how you are going to move further so this information is very important here and you can see a demonstration again from Goldstein sensation perception the movement of the car creates flow information the flow information provides information for guiding further movement.
For example you are moving very fast towards a particular obstacle or say for example there is a person crossing the road and you also immediately turned on the road you will see that the size of the person on your retina simply kind of is changing with the distance it is changing very fast with respect to whatever speed with you which you are approaching that person so these are some of the kinds of information that people have reasoned are dynamically and almost always used by our perceptual systems you can see that in that sense perception is not really a static process it is something that is changing every moment and whatever new information is produced by this change that is happening every moment also is already taken into account before you plan your next movement also Gibson said one of the things that the senses work in in communion with each other the senses work together to create this new perceptual information that you would use in order to move successfully imagine an animal who is hunting in the wild.
The Scenses
He is basically moving he’s also using the sense of smell he’s also using his sense of hearing all of that to plan his movement in order to chase the prey successfully Gibson actually produced that the five senses of vision hearing taste smell and touch they work together to produce information that actually facilitates moving around and interacting with the environment another nice example that one can take are say for example our ability to stand upright and maintain the balance of posture is basically dependent on these organs in the ear called the vestibular canals.
In the inner ear, we will talk about these when you are talking about auditory perception these vestibular canals are a kind of a canal-like structure wherein some liquid has filled the position of this liquid tells you that the head is in balance or it is not in balance so this whole ability of standing is not really only dependent on your touch or on your vision it’s also dependent on something that is going on in your ears so in that sense you can already say that we are using multiple sources of information to maintain our balance a very interesting experiment on this account was done by lee and Aronson in 1974 they actually designed what was called the swinging room experiment.
So they had a false room wherein toddlers and young very young children could stand and the walls of this room could sometimes move closer to these people or sometimes move further away or sometimes the room would swing and this kind of created a perception of movement and what was observed that the site of vision is actually found to be the more powerful source of information in order to maintain somebody’s balance and the site of and this information from the vision that is getting that the people were getting can also sometimes override other traditional sources of information like the vestibular canal.
Navigation Through The Environment
So what was happening was that when people saw the walls coming toward them they tried to move backward or they tried to compensate for that movement of the wall even though there was no signal that they are going off balance from the vestibular canal system so in that sense it was adequately established that the sense of vision has to do a lot with function like standing upright and standing straight and maintaining a particular body balance now we will take up some examples which can be examples about how people how people’s perceptual systems help them interact with the environment around them will take some naturalistic settings.
for example the setting of driving a car so land only in 1994 they wanted to study the information that people use while they are driving a car so what they did is they fitted a car with devices to record the angle of the steering wheel and also the speed and they also measured where this driver was looking while he’s driving so with the help of a video eye tracker they actually fitted a particular car with all of these gadgets and asked somebody to drive this car and when they did this experiment when some trials were done they actually found out that although drivers do look straight ahead while driving they’re not really looking at the exact spot of the focus of expansion that gibson kind of talks about they are looking not directly at the foe they are looking at a point just adjacent to the foe still in front of the car so in that sense they are not really using optic flow information per se but they are using some information from the outside environment to drive the car better let us say here you can see one of the demonstrations from goldstein so they are not really see looking at the focus of expansion they are looking at a point adjacent to the focus of expansion.
Land and Lee
Now, land and lee also wanted to study where drivers were looking while they were navigating a curved road so this was important because when you are actually navigating a curved road the focus of expansion is continuously changing so it has to be directed right in front of you but as you are changing your direction while navigating a curve the focus of expansion is also changing and so flow information is also changing so land-only were interested in this and they found out that when going around a curve the drivers are not really looking directly at the road but they are actually focusing at the tangent point on the curve of the side so they basically want to stay on the road by looking at the tangent point and not really directed at the road.
Walking
So as the drivers were not really looking directly at the focus of expansion land-only suggested that drivers are using the information in addition to the optic flow information that Gibson had specified in order to determine the direction of movement and this information could be anything something very simple like.
For example, the position of the car with respect to the lines on the center or the side of the road so is again something that tells us that people are actively engaging with the environment in order to successfully move around and navigate the environment they are taking up new sources of information, in addition, to obviously the traditional source of information in order to maintain this dynamicity of interaction so here you can see that people are actually looking at the tangent point of the curve and not really the line at the center of the road ok now another aspect that we use perception in is in walking so people walk and they navigate the environment successfully what is the kind of information they might be using here so it is believed that they are probably following something called a visual direction strategy.
So what they are doing is they are keeping their bodies pointed towards a particular target say for example if you are walking on a straight road you might kind of position your body such that the end of the road is your target and you’re kind of walking in reference to that target so if they would go off in direction say for example sometimes while walking if you’re talking to somebody or if you’re kind of thinking about something else or maybe texting nowadays you might kind of go off direction and so the target would shift to the left or the right of where it was originally and this information helps us to do what is called course correction.
So we think now we’ve got offroad and we have to come back and then maintain that position with respect to the target that we have set up so Lumis and colleagues they basically wanted to test in a series of experiments and they demonstrated this by making participants blind walk towards a target that people are able to walk directly towards a target and they kind of reach very close towards it.
for example, if you kind of tell somebody that this is your target and you have to walk towards this target and then kind of blindfold them they will still reach rather close very close to the target without even using the visual information because they’ve oriented themselves relative to that so here you can see a demonstration from Goldstein’s book that the target is this square green square in the top and the person starts at a point slightly diagonal to that and they start walking towards points one or two and they can turn towards the right at either point one or point two and the kind of you’ll see in both those in places land somewhere very close to the target they are not really off target by a lot so these set of experiments actually demonstrated that people can orient themselves while walking to a particular target and then might not be actively using a lot of visual information they might kind of orient themselves in a different way.
Wayfinding
Wayfinding is also very important somebody tells you that you have to reach there and meet me at this particular point and if you do not know the place you will be navigating for a long distance towards an object that is not already in sight the last example was when you already have an object inside and you are kind of preparing to walk towards it the other thing could be that you do not have an object inside and you kind of navigating making mental maps and moving towards that wayfinding in that sense is a slightly complex process that involves the perception of objects in the environment remembering those objects and their place in the overall scene and also judging when what direction to turn.
for example if you are remembering how to reach a particular theater or a particular coffee place from your home and it is a place you where you going for the first time you might want to remember what all shops that you found in between if there was a park or if there was an important building what we do generally is we kind of tell people that that was that there you will find this building and you can take a left or right from that particular building and then reach this place.
If you’re going alone as well you might use these strategies almost subconsciously but automatically that you remember that these are the pointers in the way of my target and I’ll remember them so that next time I can orient myself according to these points an important aspect of such navigation is these points and these points are referred to as landmarks so we remember these landmarks say for example from this particular building you have to turn left or turn right so that particular building becomes a source of important information it is associated with that you have to turn left or right during while you are reaching a particular target so these are the objects on the route that serve as cues to indicate where you have to turn where you have to get off the road so Sahar Hamid and colleagues in 2001.
They studied the use of landmarks by participants as they were negotiating a maze-like environment presented on a computer screen and while pictures of common objects were used as landmarks so they are actually in a stimulated maze they are basically taught to reach from point a to point b in a maze and at different points in the maze they use these pictures of common objects as landmarks so they will know that from this object I have to take a left to reach the end of the maze so there are two phases participants were first trained to go through the maze and they were told to travel from point a to point b in the maze in the second part they also monitored eye movements using what is called a head-mounted eye tracker.
So eye tracking measures indicated that participants spent more time looking at informative landmarks than uninformative landmarks so those landmarks that actually provided useful information about turning and reaching the target successfully were looked at much more they were given much more importance in a similar study by shnazi and Epstein it was shown that when subjects had learned a particular route they were more likely to recognize pictures of buildings at those decision points so you might say for example remember that that is a particular crossing from which I have to take a turn.
so you will remember how that crossing looks what are the important shops or posters on that crossing similarly the participants in this study also remembered pictures of landmarks at those particular pictures at those particular landmarks that were informative for reaching towards a particular target also when these participants were taken in an fmra scanner it was found that the brain responses in the navigational areas of the brain which are like the para hypoglossal gyrus hippocampal gyrus and the retro spiral cortex the responses of these particular brain areas was much larger to these informative landmarks than to non-informative landmarks or non-decision building points so this kind of tells us that the brain is constantly keeping track of wherein we are interacting with the environment what are the sources of important information.
So, that it can plan and it can be prepared for action for navigation for moving around the environment you’re seeing that you’re not just following instructions even if somebody gives you a map and says that you reach from point a to point b and you kind of follow these steps you are obviously all the time gaining much more information by really taking the car or your cycle or walking around that road here is this example of the brain you can see the Hippocampus the Parahippocampal gyrus and the Retrosplenial cortex.
Interacting With Objects
Now we have seen how movement actually within an environment can facilitate or influence perception one of the most salient movements we actually perform is when we reach out and grasp objects say for example if there is a cup here reach out to the cup and i pick up the cup to drink from it an important aspect about reaching and grasping behavior is this concept of affordances again you might want to revise the article on Gibson’s ecological approach of perception because that is what Gibson had already talked about so what are affordances, affordances are basically things that a particular object renders or offers you say
for example, if there is a ball that does not have any edges you might want to pick the ball itself if there is an object which has a handle you might want to pick that object by that handle or say for example if there is an object which is which has a plane surface and a ford sitting you might sit on that as well you will see people doing this kind of behavior all around yourselves why are they able to do that because they understand that this object affords this kind of actions a lot of times you will see that if you do not find a hammer or if you do not find a screwdriver you might use unconventional tools to hammer in a nail or to open and try and try and open a particular screw.
So why are we able to do this Because we understand that objects have particular affordances and that objects can be interacted with in a particular manner and in that sense we are kind of always prepared to use this information in our interaction with these objects so this basically could imply that perception of an object then not only includes information about physical properties such as shape size color and orientation it also might enable the person to recognize the object but our perception also includes information about how this object has to be interacted with or how this object has to be used one of the very interesting studies about affordances was done by Glenn Humphries and jane Reduck and they basically were studying a patient named mp and this patient mp had damage to his temporal lobe and this damage to the temporal lobe had led to his led to an impaired ability to remember to name objects so this person would not be able to name an object if you show him.
so mp was during this course of this study I was given two kinds of cues one of the queues will be the name of the object and the other queue would be the use of the object say for example i could tell him that this is a cup or I could tell him that this is an object that I can drink from and then what will be done is he will be shown 10 or 15 different kinds of objects and he was told to press the key as soon as one of the already shown object is presented it was found that mp identified the object more accurately and more rapidly.
when the queue given referred to the function of the object so it seems that they basically concluded that mp was using the information about the use of that particular object or affordance of that particular object in recognizing the object and not really just the name of the object it tells you something important it tells you that there are these very specific ways in which we interact with these objects and these specific ways help us store these objects in a much better way in our semantic memory in our knowledge of the world.
How the brain helps us in interacting with the environment?
and this basically was explored in a very interesting study by a younger leader and Mishkin what they did was they wanted to study a monkey’s ability to identify a particular object or to determine the location of a particular object so angular in Michigan presented a monkey with two tasks –
- An Object Discrimination Problem.
- Landmark Discrimination Problem in the Object Discrimination Problem.
The monkey was shown one object say for example a rectangular solid and then was presented with two choice tasks he was presented with the object that the monkey has already been shown and another stimulus that he had to differentiate this object from now the monkey was supposed to push aside the target object and then he would get some reward that was placed under this target object so you can see the setup is something like this.
So the target object could be the rectangle and the stimulus object could be the triangle and if the monkey successfully moves aside the rectangle which is the target object then the monkey will get the food as a reward the other kind of task was that the monkeys basically had to move the food well cover that is closer to a tall cylinder so you see here there are two kinds of covers one of them contains food but the monkey has to identify the cover that is closer to the cylinder because that is where the food will be filled in.
so the monkey has to basically do is determine the position of the object because the identity of the two objects is pretty much the same behavioral experiments with these monkeys basically showed that object discrimination problem was particularly difficult for monkeys now the second so I will just kind of go back and I will tell you that the second phase of this experiment was the ablation of the monkey’s brain and ablation of monkey’s brain was basically in one set of the monkeys the temporal lobes were removed in another side of the monkeys the parietal lobes were removed so this was the second important part and then when the behavioral testing was done it was found that object discrimination became an important became a very difficult task for monkeys who had got their temporal lobes removed.
Now on the basis of this result, it was implied that the pathway that reaches the temporal lobes is basically the pathway that is responsible for object identification so the temporal lobes in that sense can be implicated in the identification of objects anger leaders in Michigan basically called this pathway.
Starting from the straight cortex in the occipital lobes as the what pathway monkeys who had their parietal lobes removed had difficulty solving the landmark discrimination problem now this indicated that the pathway leading to the parietal lobe is responsible for determining an object’s location ungulated in Michigan called this pathway leading from the striate cortex to the parietal lobe as the weir pathway so this is the pathway which is telling them the spatial location of objects the other pathway the what pathway is basically telling them what these objects are remembered that you will need both these kinds of information if you really want to navigate successfully with the environment here you can see the dorsal and the ventral pathways the vare pathway is the dorsal pathway and the ventral pathway is the what pathway.
So in a simple task of actually reaching and grasping a cup one could assume that what pathway would be involved in the initial perception of the cup and the vare pathway will tell you where the cup is and how much my hand should move away from my body in order to be able to grasp the cup and drink from it so in that sense we are kind of using information from both these sources and we are kind of integrating this information in order to successfully interact with this object that is the cup on my coffee table.
now we kind of try and summarize whatever we have learned with respect to these approaches so we have seen that Gibson’s approach was basically pushing for perception for action while David Marr was more concerned about perception for recognition both of these approaches are more or less correct in their own way but you have to try and integrate both of these approaches the findings and the perspectives from both of these approaches to understand perception as a whole cognitive function it seems that in some way the idea of both these ventral and dorsal pathways can of echo the similar idea.
so the dorsal pathway is the perception for action kind of pathway and while the ventral pathway is the perception for recognition kind of pathway that David mar was talking about so while these two streams may seem slightly different from each other and independent from each other I am sure you would appreciate that if you have only one of this information you will not be able to successfully interact with the environment so we necessarily and definitely need information from both of these pathways in order to understand and navigate the environment successfully so Gibson’s approach of affordances emphasizes that we might need to detect what things are for rather than what they actually are.
so affordances are linked to actions and to the dorsal stream basically that appears to be ideally suited for providing that kind of information also if you remember we saw that Gibson said that there’s no role of memory in perception and as such the dorsal stream anyways has very little storage and so this also confirms that the dorsal pathway works pretty much as Gibson was assuming in contrast if you see the ventral stream appears to be ideally suited to the role of recognizing objects.
so it is specialized in analyzing the kind of fine detail that David mar was concerned with the textures and the gradients and those kind of things and this information will be used in discriminating between objects also as it seems that we draw on our existing knowledge to understand and identify objects and so the dorsal the ventral stream basically kind of is drawing from that source as well it is also the ventral stream is also slower than the dorsal stream which is conducive to the fact that no immediate action is required so if i say for example through a chalk towards somebody who is not really looking at me the first thing that you do is you kind of grasp the chalk irrespective of recognizing whether it is a chalk or a pointed object or something very light ok because that information will automatically require immediate action and in that sense the dorsal pathway will be the one which will be required to act so to address these different kind of concerns and these two slightly different sources of information norman and Nizar in 1994 suggested what is called the dual processing approach.
So, they said that there appears to be evidence that ventral stream is primarily concerned with recognition while the dorsal stream is primarily concerned with visual behavior the ventral stream as it said and as it has been found has been found to be generally better at processing fine detail while the doses system is more suited to processing motion the ventral system is knowledge based and uses stored representations from memory while the dorsal system appears to have only very short term storage that is required to act and finish a particular task also the dos system receives information much faster than the ventral system and we are much more conscious of the ventral than the dorsal system again if you take the example of somebody throwing a chalk at you or something that you have not seen you will see that the ventral system does not require that kind of processing it first needs to take evasive action so it has been suggested on the basis of these things that the ventral stream recognizes objects and is object centered in its approach while the doses stream is action oriented and it receives a viewer centered frame of reference ok so it moving slightly further.
So, norman kind of tries and defines the two as synergistic systems basically working in an interconnected rather than an independent fashion burst in Carlton they provided an illustration of the interaction between ventral and dorsal streams when people are learning new tasks so the work fits shows that in the early stages of learning a skill when you are not really aware of what has to happen.
For example, if you are just learning driving of a car you will just first get very familiar with the consoles of the car the gear and other pedals and those kinds of things in that part you’re kind of using the ventral system a bit more once the skill is formalized once you get acquainted with the particular skill then what you might shift onto is a highly practiced skill wherein the dorsal stream might be more useful.
so your kind of just doing things for action say for example after you’ve learned driving a car for a particular amount of time then you’re not really always looking at what the gear is and where the pedals are you already know that they are there and they have to serve this function and whenever you need to change the gas your hand automatically goes on to the gas and you can change the gas in order to move forward ok so this is the end of this link between perception and action we tried and looked at different aspects of how our movement in the environment facilitates or helps out our action and also how to say for example information from the objects can help us modulate or influence our interaction with the environment thank you.
Read also: