ORIGINAL PAPER Mono and Trimethine Cyanines Cyan 40 and Cyan 2 as Probes for Highly Selective Fluorescent Detection of Non-canonical DNA Structures Vladyslava B. Kovalska & Mykhaylo Yu. Losytskyy & Sergiy M. Yarmoluk & Irit Lubitz & Alexander B. Kotlyar Received: 5 May 2010 / Accepted: 11 August 2010 / Published online: 1 September 2010 # Springer Science+Business Media, LLC 2010 Abstract Two of earlier reported dsDNA sensitive cyanine dyes—monomethine Cyan 40 and meso-substituted trime- thine Cyan 2 were studied for their ability to interact with non-canonical DNA conformations. These dyes were characterized by spectral-luminescent methods in the presence of G-quadruplex, triplex and dsDNA motifs. We have demonstrated that Cyan 2 binds strongly and preferentially to triple- and quadruple-stranded DNA forms that results in a strong enhancement of the dye fluores- cence, as compared to dsDNA, while Cyan 40 form fluorescent complexes preferentially only with the triplex form. Highly fluorescent complexes of Cyan 2 with DNA triplexes and G-quadruplexes and Cyan 40 with DNA triplexes are very stable and do not dissociate during gel electrophoresis, leading to preferential staining of the above DNA forms in gels. The data presented point to the intercalation mechanism of the Cyan 2 binding to G4- DNA, while the complexes of Cyan 40 and Cyan 2 with triplex DNA are believed to be formed via groove binding mode. The Cyan dyes can provide a highly sensitive method for detection and quantification of non-canonical structures in genome. Keywords Cyanine dyes . Fluorescence . Non-canonical DNA Introduction In living organisms mostly genomic DNAs exist in a double-stranded (ds) helical form, but occurrence of some other conformations also seems possible. The guanine-rich nucleic acid sequences are known to fold into four-stranded (G-quadruplexes, G4-DNA) or triple-stranded (triplex) structures in which the nucleotide bases are connected by means of Hoogsteen base-pairing bonding (Fig. 1). It was shown that in the human genome, more than 300,000 distinct sites can potentially form G4-DNA structures [1, 2]. This attracted significant attention of the research commu- nity to the problem of the possible role of triplex and quadruplex DNA motifs in gene regulation and genomic stability [3–6]. Therefore, development of a specific probe for distinguishing quadruplex or triplex conformations from a canonical ds one may have a significant scientific and practical importance. Among the dyes proposed for the detection of G4- DNA regions, porphyrin dyes have been found to give much stronger fluorescent response in the presence of G- quadruplexes compared to dsDNA [7–9]. Since the 1990s cyanine dyes have been intensively applied for DNA and RNA detection and quantification [10]. SYBR Green I [10], which belongs to the family of monomethine cyanine dyes [11], is considered to be one of the most sensitive dsDNA probes up so far. DNA-binding dye, Thiazole Orange (TO) has been shown to strongly and selectively bind to G-quadruplexes and triplexes [12, 13]. Chemical conjugation of TO with Bisquinolinium, a G- quadruplex specific ligand, yielded selective fluorescent V. B. Kovalska (*) : M. Y. Losytskyy : S. M. Yarmoluk Department of Combinatorial Chemistry, Institute of Molecular Biology and Genetics of the National Academy of Sciences of Ukraine, 150 Zabolotnogo St., 03143 Kyiv, Ukraine e-mail: v.b.kovalska@imbg.org.ua I. Lubitz : A. B. Kotlyar (*) Department of Biochemistry, George S. Wise Faculty of Life Sciences, Tel Aviv University, Ramat Aviv 69978, Israel e-mail: s2shak@post.tau.ac.il J Fluoresc (2011) 21:223–230 DOI 10.1007/s10895-010-0709-y