I read about "topological crossover" in this paper, Table 1. What does this refer to?

You may not want to know. It really depends on how good your math background is.

A topological crossover is a class of representation independent operators that are well defined once a notion of distance over the solution space is defined [1].

Let's first start with the definition of Topology.

Topology is the mathematical study of shapes and topological spaces. It is an area of mathematics concerned with the properties of space that are preserved under continuous deformations including stretching and bending, but not tearing or gluing. This includes such properties as connectedness, continuity and boundary [2].

Later on in the paper they start to mention transverse topology which leads to Differential Geometry.

In mathematics, transversality is a notion that describes how spaces can intersect; transversality can be seen as the "opposite" of tangency, and plays a role in general position. It formalizes the idea of a generic intersection in differential topology. It is defined by considering the linearizations of the intersecting spaces at the points of intersection [3].

To understand the underlining math concepts of this paper, you may need a rigorous background in mathematics. I would check the authors backgrounds to see if they are Physicists or Mathematicians.

Most students in mathematics generally take two courses in introductory real analysis and at least one course in abstract algebra before taking topology. The standard book for a first course in topology is by Munkres.

A book you may want to look into is Topology in Molecular Biology. However, the author is a mathematician, so if you are not, you may need to start with courses in analysis, topology, differential geometry before you tackle this book. I have never seen the book so I don't know how difficult it will be but here is the about the author from amazon:

M.Monastyrsky is a professor of mathematics and theoretical physics at the Institute of Theoretical and Experimental Physics, Moscow, Russia, graduated the mathematical department of Moscow state university(1967), author of numerous papers in diferent fields of mathematics and theoretical physics including the monograph :Topology in Gauge Fields and Condensed Matter (Plenum,1993 ) and two other books:Riemann,Topology Physics, Birkhauser, Boston, 1999, 2nd ed. and Modern mathematics in the light of Fields medals, AKPeters, 1997.I gave invited plenary talks on many conferences and was the visiting professor of many universities and institutes including IAS(Princeton), Harvard, Princeton and Yale Universities (USA), Cambridge and Oxford Universities (Great Britain), IHES and Universite de Paris (France).