/2/2 Modeling Negative Priming Using EEG-Data Hecke Schrobsdorff [1,2] , Matthias Ihrke [1,3] , J¨ org Behrendt [1,3] , Marcus Hasselhorn [1,3] , J. Michael Herrmann [1,2] [1] BCCN G¨ ottingen, Bunsenstrasse 10, 37073 G¨ottingen [2] University of G¨ ottingen, Institute for Nonlinear Dynamics, Bunsenstr. 10, 37073 G¨ottingen [3] Georg-Elias-M¨ uller-Institute, Section for Educational and Developmental Psychology; University of G¨ ottingen, Waldweg 26, 37073 G¨ ottingen hecke|michael@nld.ds.mpg.de, mihrke|jbehren1|mhassel1@uni-goettingen.de göttingen Introduction Priming effects are characterized by a modified reaction time depending on the stimulus sequence. While a reduction in reaction time as occurring in positive priming is a relatively well-understood phenomenon, the opposite effect is experimentally much less tangible. However, negative priming is interesting as a mechanism in selective attention. Explanations of the negative priming effect have been discussed controversially over the past 20 years, yet no consensus among the researchers is in sight. One of the reasons beside a general difficulty to normalize experimental conditions is the lack of a concrete computational formulation of the psychological theories which would make cross testing on a quantitative level possible. In order to deal with the question of the scope of applicability of the numerous theories, we developed a general model which attempts to incorporate all major theories. As the model was designed with the intention to apply it to diverse paradigms, it comprises a recent solution for the feature binding problem as well as a minimalistic version of episodic memory. The general model makes concrete statements at which locations and point in time the different theories apply and to what extent they contribute to negative priming. We present different paradigms of behavioral negative priming experiments and review how the major theories explain decision making and the sources for negative priming. As a crucial component of the general model we shortly introduce the implementation of a feature binding model with spiking neurons. After the presentation of the model structure together with its implementation, we point at EEG findings and how they can be used to concretize parts of the arbitrarity of every modeling approach. Finally we show the models behavior in the different priming conditions. The Psychological Effect Negative Priming Paradigms Negative Priming Experiments show stimuli consisting of a target and a distractor . Depending on the paradigm these stimuli can be pictograms, dots at specific locations, letters etc. Negative priming was found also by presenting auditory and tactile stimuli. Every paradigm assigns a stimulus feature that characterizes the role as target or distractor, color in these examples. Also, a feature that defines the correct response, here location, letter identity, object identity and match of the object identity to a comparison word is assigned by instruction. location priming flanker priming D D C A D B C p identification p "Ball" comparison match mismatch BALL Continuous Stimulus Presentation NP time reaction time reaction response stimulus interval PP CO stimulus onset ⋆ A continuous stream of trials is processed. ⋆ Two subsequent displays form a prime-probe pair. ⋆ The reoccurrence of prime objects defines the priming con- dition of the probe trial. ⋆ This method has been optimized experimentally for strong priming effects and short trial duration to minimize experi- ment duration. ⋆ The reliability of NP is much weaker in comparison to PP. ⋆ NP tasks show a large inter- and intra- individual variability. PP PP2 control NP NP2 Trial Condition Reaction Time [ms] 0 200 400 600 800 1000 Negative Priming Theories Distractor Inhibition [3] — In this controversial model, NP is produced by an inertia effect of the underlying artificial neural network. In order to distinguish target from distractor, the network inhibits the distractor. When the input is switched off, the remain- ing inhibition produces an inhibitory rebound of the distractor. To respond to the former distractor in the probe trial takes longer as the activation has to rise from below baseline. 0 1 pseudo activation target distractor time reaction time trial onset Do not respond Do not respond "Bank" "Ball" "Bank" retrieval conflict NP retrieval conflict Prime Probe Episodic Retrieval [7, 2] and its variants Temporal Discrimination [6] and Feature Mismatch [8] — The trial onset triggers a retrieval of the former trial from episodic memory. A ’do not respond’-tag, that was attached to the distractor in the prime episode is retrieved as well. This tag is conflicting with the current task to respond to the former distractor, it has to be removed and thus causes the delay observed in negative priming conditions. Recent advancements state that similarities of prime and probe episode trigger mainly the retrieval of the prime reaction. The more similar the trials are, the stronger is the retrieved representation. In classical negative priming experiments NP-trials always change the reaction whereas positive priming always requires the repetition of reaction, thereby producing the classical effects. Imago-Semantic Action Model [10] — The ISAM decides between target and distractor via a threshold that adapts to a global activation level. The target is singled out by a semantic feedback loop. A reaction is triggered as soon as only one object representation is above threshold. A computational implementation with rate equation shows a rich and promising behavior of the ISAM also in nonstandard situations. of Relevance Automatic Rating Post-Hoc Rating of Relevance Pattern Recognition Semantic Analysis Semantic Transcoding Adaptive Threshold Sensory Input Situational Acuteness Space of Possible Actions Dual Mechanism Hypothesis [5] — Negative Priming might be produced by an interplay of several mechanisms, depending on stimuli, strategies, task etc. Therefore, in settings where memory strategies are an inherent part of the paradigm, memory mechanisms might be more prominent whereas in other situations the inhibition view might provide a deeper insight. A Feature-Binding Model [9] Problem PFC V4 IT Visually perceived objects are decomposed into their diverse features as perception travels up the visual pathway. The combination of these features into a coherent representation of the perceived entity is generally called feature binding. Due to the combinatorial explosion, the binding has to be organized in a very flexible way. We recently developed a feature binding model that is based upon localized excitations in layers of spiking neurons. The binding occurs by dynamically linking localized excitations in different chains via phase locking. Model ⋆ Three one-dimensional layers of leaky integrate-and-fire neurons. ⋆ All layers with a locally excitatory, globally in- hibitory coupling kernel. ⋆ Stable localized activation whose position en- codes for features. ⋆ Weak all-to-all excitatory coupling between both feature regions to the binding layer and vice versa 0 1 2 3 4 5 6 7 8 9 10 20 40 60 80 100 120 0 1 2 3 4 5 6 7 8 9 10 20 40 60 80 100 120 0 1 2 3 4 5 6 7 8 9 10 20 40 60 80 100 120 IT PFC V4 time external input external input σ σ σ σ Results ⋆ The feature binding model is able to dynamically link localized excitations. ⋆ These activations are formed via subthreshold perceptual input. ⋆ Due to symmetry breaking the linking layer assigns a local position for the link. General Negative Priming Model Even though an implementation based on spiking units is generally possible, we limited the current approach to an abstract rate code model in order to minimize effects arising naturally from more complex numerical specificities. It is thus possible to focus more on the specific effects arising from the theoretical ideas. The visual characteristics of the experimental stimuli strongly affects priming effects. In order to achieve a detailed representation of the stimuli, they are split into different features. This approach permits a separate identification and handling of features that are responsible for target identification or response generation, respectively. A flexible feature binding mechanism stores the information about the entity of objects as well as the synaptic connection between feature layers, balancing the connected activations. A forced decay of similar but nonmatching bindings is proposed by the ISAM. All theories agree that response generation is based on semantic representations. Depending on the paradigm, concrete mappings from activations in the semantic layer to actual responses are given by a central executive unit that controls task specificities. A finished trial is stored in episodic memory. The similiarity of the current and the memor- ized percept determines the impact that the memory re- presentation exercises on the current trial. Concrete Implementation Feature Layers Feature activations c, s, w rise with τ rise to- wards input I . In the absence of input, they passively decay with τ decay . Via the bindings activation between features of the same object balance. The feature instance that defines a target as such is amplified with gain γ target . With retrieval strength r times strength of the memory trace m the old episode c old is retrieved. dc dt = τ rise/decay (I - c)+ b(s - c) · c +γ target + r · m · (c old - c) Binding Layer Bindings are inititated at stimulus onset, their synaptic strength b adapts with time constant τ b towards a maximum strength, if p =1, i.e. the binding is currently per- ceived. Bindings that share several, but not all features cause a forced decay db dt = τ b (p - b) -  similar bindings τ f · ǫ · b Semantic Layer Input to the semantic layer S from shape s and word w layer with synap- tic strengths σ s, S ,σ w, S . dS dt = σ s, S (s - S )+ σ w, S (w - S ) Threshold θ adaptation with time constant τ θ to an average activity. dθ dt = τ θ · 1 2  (S - θ ) Action Layer As a winner-takes-all mechanism for ac- tion selection, we chose the Fisher Eigen Equation. The activation of an action a competes with adaptive fitness a f that ap- proaches fixpoints f 0 defined by the central executive. Between trials, actions are in- hibited by assigning a positive fitness to the decision not to react a no . da dt = τ a (a f -  a f · a + a f, no · a no ) da f dt = τ a f (f 0 - a f ) Episodic Memory After a response, the entire episode, i.e. ac- tivation variables of the feature layers their bindings and semantic activations as well as the action fitnesses, are stored in episodic memory. The memory trace is assigned an inital strength m that decays with time con- stant τ m . dm dt = -τ m During perceptual input, the current per- cept is compared to the last stored episode. The resulting strength of retrieval r is a normalized sum of the number of match- ing features # feature and correct bindings # correct binding , weighted by the current ac- tivation of the percept. r =  o∈{percept}  B ∈{bindings} a·(# o,B feature +# o,B correct ) Results from EEG Recordings Lateralized Reduced P300 Amplitude ⋆ left hemispheric ⋆ P300 reduction ⋆ for both NP, PP P5 P6 Revelation of Hidden Time Markers A large distance between objects and comparison word in the comparison paradigm provokes a saccade at the time when the target object is classified. The saccade is detected by EOG electrodes thus indicating the end of the classification process. -334 -167 0 167 334 EOGru Time (ms) -1000 -500 0 500 1000 1500 100 200 300 400 500 600 700 800 Given this setting, NP is produced by the first sequence, whereas PP becomes only significant in the time interval between saccade and reaction. Response-Latency Corrected Averaging See workshop on Wednesday, July 11, 15:20. ”Synchronization of brain signals: what is real, what is not” Frontal PSW Differences ⋆ late positive complex ⋆ more positive for NP ⋆ more negative for PP FPZ Recording Setup ⋆ 64 channels ⋆ extended 10-20 system ⋆ four EOG electrodes -50 0 50 100 -50 0 50 100 -60 -40 -20 0 20 40 60 80 100 ECGlo EOGlu Y F7 F5 AF7 FT7 FC5 F3 AF3 Fp1 X FC3 T7 C5 F1 Fpz AFz C3 FC1 TP7 CP5 TP9 Fz EOGru Fp2 C1 CP3 AF4 P7 F2 P5 EOGro AF8 CP1 P3 Z Cz F4 FC2 PO7 F6 P1 CPz F8 PO3 FC4 C2 Pz FC6 FT8 CP2 C4 POz Oz P2 C6 T8 CP4 TP10 PO4 P4 CP6 TP8 PO8 P6 P8 Implications for Modeling ⋆ Positive and negative priming share early mechanisms. ⋆ Their processing differs in later control stages. ⋆ EEG-data can reveal information about the time course of the processing. ⋆ Single trial fitting can identify stages re- sponsible for variance Modeling Scope Activation Traces 1.95 2 2.05 2.1 2.15 2.2 2.25 2.3 2.35 2.4 x 10 4 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Shape Control 1.7 1.75 1.8 1.85 1.9 1.95 2 2.05 2.1 2.15 x 10 4 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Shape Negative Priming 2.2 2.25 2.3 2.35 2.4 2.45 2.5 2.55 2.6 x 10 4 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Shape Positive Priming Reaction times condition control NP PP mean reaction time 1018.57 1055.0 989.38 effect —– -36.43 29.2 variance of rt 17.3 47.96 49.8 For a simulated experiment with 40 trials in the word-picture comparison paradigm with an RSI = 1500ms, the general model produced the following reaction times: Localization of Predicted Effects 1.9 1.95 2 2.05 2.1 2.15 2.2 x 10 4 0 0.2 0.4 0.6 0.8 1 1.2 Color Activation of the color green is boosted as it de- fines the target object. The plus of activation is passed on to the shape layer via bindings thereby singling out the target ob- ject in semantic space. 1.65 1.7 1.75 1.8 1.85 1.9 x 10 4 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Semantic In the semantic layer, the threshold adapts such that it settles between first and second strongest activation, when only two activations show signifi- cant activation. Other- wise it surmounts all ac- tivations. 1.5 2 2.5 3 x 10 4 0 1 2 3 4 5 6 7 8 x 10 -3 Episodic memory is sub- ject to a decay over time. Depending on the similar- ity between memory and percept and the activation strengths of the feature activations, the retrieval strength takes on different values. 2.23 2.24 2.25 2.26 2.27 2.28 2.29 2.3 2.31 2.32 2.33 x 10 4 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Action The Fisher-Eigen- Equation stands out by an exponential behavior of different variables that share common recources, here the constant overall activation. An action first has to inhibit the ’do not respond’-variable. Conclusion ⋆ In an exemplary interdisziplinary project that combines computational neuroscience and cognitive psychology, we interactively developed elaborated experimental paradigms to- gether and with the aid of a comprehensive model. ⋆ The model incorporates various components: perception, attention, memory, semantic representations and action selection, thereby allowing us to cross test concurring theories for negative priming. ⋆ Reaction time differences in negative priming conditions emerge due to an interplay of all model components. ⋆ A model based analysis of EEG data eases the finding and interpretation of neural cor- relates of negative priming. Outlook ⋆ In order to cross test the different theories, parameters will be collated to single setscrews to tune the impact of a single theory on the model behavior. ⋆ Peculiarities of certain features have to be translated into topologies of the specific feature layer to apply the model to a broader range of stimuli and paradigms. ⋆ Integrating effects of cognitive aging into the model. References [1] E. Fox. Negative priming from ignored distractors in visual selection: A review. Psychonomic Bulletin & Review, 2(2):145–173, 1995. [2] Frings, C., Rothermund, K. and Wentura, D. (in press). Distractor repetitions retrieve previous responses to targets. Quarterly Journal of Experimental Psychology. [3] Houghton, G. and Tipper, S. P. (1994). A dynamic model of selective attention. In Dagenbach, D. and Carr, T., editors, Inhibitory mechanism in attention, memory and language, pages 53–112, Orlando, FL. Academic Press. [4] B. Kabisch. Negatives Priming und Schizophrenie - Formulierung und empirische Untersuchung eines neuen theoretischen Ansatzes. PhD thesis, Friedrich-Schiller-Universit¨ at, 2003. [5] May, C. P., Kane, M. J., and Hasher, L. (1995). Determinants of negative priming. Psychological Bulletin, 118(1):35–54. [6] Milliken, B., Joordens, S., Merikle, P. M., and Seiffert, A. E. (1998). Selective attention: A re-evaluation of the implications of negative priming. Psychological Review, 105(2):203–229. [7] Neill, W. T. and Valdes, L. A. (1992). The persistence of negative priming: Steady-state or decay? Journal of Experimental Psychology: Learning, Memory, and Cognition, 18:565–576. [8] Park, J. and Kanwisher, N. (1994). Negative priming for spatial locations: identity mismatching, not distractor inhibition J Exp Psychol Hum Percept Perform, 20:613–623. [9] Schrobsdorff, H., Herrmann, J. M., and Geisel, T. (2007). A feature-binding model with localized excitations. Neurocom- puting, 70(10-20):1706–1710. [10] Schrobsdorff, H., Ihrke, M., Behrendt, J., Hasselhorn and M., Herrmann, J. M. (2007). A Computational Approach to Negative Priming. Connection Science, in press. Acknowledgments The authors like to thank ⋆ Torsten W¨ ustenfeld ⋆ Henning Gibbons ⋆ Ralph Meier ⋆ Miguel Valencia Ust´ arroz ⋆ Theo Geisel ⋆ Tobias Niemann This work was funded by the BMBF in the framework of the Bernstein Center for Computa- tional Neuroscience G¨ ottingen project C4. Grant number 01GQ0432. 1.5 2 2.5 3 x 10 4 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.5 2 2.5 3 x 10 4 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.5 2 2.5 3 x 10 4 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.5 2 2.5 3 x 10 4 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 x 10 -3 1.5 2 2.5 3 x 10 4 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.5 2 2.5 3 x 10 4 0 1 2 3 4 5 6 7 8 x 10 -3 1.5 2 2.5 3 x 10 4 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Bus Perceptual Input NO Response target: green task: compare Central Executive color Feature Layers Binding Layer Semantic Layer Action Layer Episodic Memory shape word Biggest Poster Award and Best Poster Award won at CNS*2007