Neuroscience Vol. 53, No. 2, pp. 465473, 1993 Printed in Great Britain 0306-4522/93$6.00 + 0.00 PergamonPress Ltd © 1993IBRO 5-HYDROXYTRYPTAMINE 1 RECOGNITION SITES IN RAT BRAIN: HETEROGENEITY OF NON-5-HYDROXYTRYPTAMINEIA/lC BINDING SITES REVEALED BY QUANTITATIVE RECEPTOR AUTORADIOGRAPHY A. T. BRUINVELS, J. M. PALACIOS* and D. HOYER~" Sandoz Pharma Ltd, Preclinical Research 360/604, CH 4002 Basel, Switzerland Abstract--Quantitative in vitro receptor autoradiography was used to characterize the [3H]5-hydroxytryp- tamine binding sites which are not sensitive to 8-hydroxy-2-(di-N-propylamino)tetralin,mesulergine and serotonin-5-O-carboxy-methyl-glycyl-tyrosinamide, in a non-5-hydroxytryptamine~A/m/~C/iD receptor population, in rat brain. Displacement of [3H]5-hydroxytryptamine [in the presence of 100nM 8-hydroxy-2-(di-N-propyl- amino)tetralin and mesulergine, to block 5-hydroxytryptamine~A and 5-hydroxytryptamine~c sites] with (-)pindolol, 5-hydroxy-3(4-1,2,5,6-tetrahydropyridyl)-4-azaindole,sumatriptan and serotonin-5-O- carboxy-methyl-glycyl-tyrosinamide yielded complex competition curves suggesting the presence of 5-hydroxytryptaminelB and 5-hydroxytryptaminetD sites and an additional [3H]5-hydroxytryptamine- sensitive component in rat brain. The non-5-hydroxytryptamine~^/~B/tC/l D binding sites were localized in olfactory tubercle, several nuclei of the amygdala, bed nucleus of the stria terminalis, caudate-putamen, CA3 field of the hippocampus, the frontoparietal cortex (motor area) and parts of the striate cortex. All the drugs used had low affinity for the unknown recognition site, which therefore might be comparable to the [3H]5-hydroxytryptaminebinding site reported to display low affinity for sumatriptan and 5-carboxamidotryptamine in the brains of various species, the so-called 5-hydroxytryptamine m site. A comparison of the density of sites labelled with [125I]serotonin-5-O-carboxy-methyl-glycyl-tyrosi- namide (representing 5-hydroxytryptamine m and 5-hydroxytryptaminelD sites) and [3H]5-hydroxytrypta- mine (under the conditions mentioned above) showed the density of [3H]5-hydroxytryptamine recognition sites to be higher in some structures. It is concluded that in addition to the 5-hydroxytryptamine m and 5-hydroxytryptamineto, an uniden- tified 5-hydroxytryptamine binding site is labelled with [3H]5-hydroxytryptamine,in the presence of 5-hydroxytryptamine~A and 5-hydroxytryptamine~c blocking agents, in several structures of the rat brain. Since the compounds tested displayed very low affinity for this site, the availability of high-affinityligands would certainly facilitate further characterization of this [3H]5-hydroxytryptamine recognition site, which may correspond to the 5-hydroxytryptaminetEsite. Although classical pharmacological, physiological and biochemical studies suggested several subdivi- sions of the 5-hydroxytryptamine (serotonin, 5-HT) receptor family,3'7 recent molecular cloning tech- niques provide evidence for even more 5-HT receptor subtypes. 11'12'18'2°'21 At present, the existence of four 5-HT receptor subclasses is suggested7'25 (5-HTI, 5- HT2, 5-HT 3 and 5-HT4) and each receptor subclass is composed of several receptor subtypes. Initially, radio- ligand binding in brain homogenates or on whole-tis- sue sections demonstrated that the 5-HT~ binding site, which is the only subclass that is labelled by *Present address: Departemento de Neuroquimica, C.I.D., C.S.I.C., Jordi Girona 18-26, Barcelona 08034, Spain. tTo whom correspondence should be addressed. Abbreviations: CP 93129, 5-hydroxy-3(4-1,2,5,6-tetrahy- dropyridyl)-4-azaindole: 5 - C T , 5-carboxamidotry- ptamine; GTI, serotonin-5-O carboxymethyl-glycyl- trosinamide, 5-HT, serotonin, 5-hydroxytryptamine; ICYP, iodocyanopindolol; 8-OH-DPAT, 8-hydroxy-2- (di-N-propylamino)tetralin. [3H]5-HT, could be subdivided in the 5-HT~A and 5-HTm,2a the 5-HTIcz2 and the 5-HT,D 13 receptor subtypes. Indeed, these 5-HT~ receptor subtypes have been cloned; l~ however the situation of the 5-HTm/lO receptor subtypes is more complex than originally expected, lz 5-HTIo binding sites, initially thought to be the mammalian homologue of the rodent 5-HT, B receptor, were shown to be a composite of 5-HTIo, and 5-HTIo# sites which have similar 5-HT~o phar- macological profiles in man (and other mammals) but very distinct pharmacological characteristics in rodents, where the 5-HT~o, receptor shows a 5-HT~o- like pharmacology and the 5-HT~o~ receptor shows the 5-HTIB pharmacological profile.6'1°'12 For the 5-HTIA receptor subtype several selective drugs are available, of which 8-hydroxy-2-(di-N- propylamino)tetralin (8-OH-DPAT) is the most widely used) 4 The tritiated compound labels exclu- sively the 5-HTIA site 8 and low concentrations of the drug can be used to saturate the 5-HTIA binding site. 23 The 5-HT~c/2 antagonist [3H]mesulergine labels 465