Understand the Neo4j Cypher and OGM type system

There are several layers of abstraction involved getting values of a given type into Neo4j and out of the database again. Chosing one technology to write a value and another to retrieve it, can lead to hard to spot errors. This tip demonstrate some combinations of drivers that work well to together and explains why certain values end up in the database with a different type than on the Java-side of things.

1. Problem

You have a node with a numeric value in your database. On the client side, you’ll expect an int or java.lang.Integer but are getting back a long or java.lang.Long. from you connection

2. Solution

There is no "one-size-fits" all solution to this problem, but merely an explanation and a recommendation. The database itself supports a lot of different types like byte, short, int, long, String, arrays thereof and many more. This types are however database internal. You interact with these internal types as soon as you interact with an embedded database without using a driver. It doesn’t matter if you make direct API calls or interact through Cypher with the database.

The term driver is used in multiple places: First and foremost it’s a library that implements a certain protocol to connect a client to a database. The Neo4j-Java-Driver is such a driver. It implements the Bolt protocol. Neo4j-OGM also uses the term driver but here, the OGM-Bolt-driver sits on top of the Java-Driver. The OGM-embedded-driver uses the embedded API directly.

Look at Figure 1 to see how the the Java driver, embedded database, OGM and finally work together:

Figure 1. Moving parts in type conversion

Cypher itself exposes the following types:

Table 1. The Cypher type system
Cypher Type	Parameter	Result
`null`*	✔	✔
`List`	✔	✔
`Map`	✔	✔
`Boolean`	✔	✔
`Integer`	✔	✔
`Float`	✔	✔
`String`	✔	✔
`ByteArray`	✔	✔
`Date`	✔	✔
`Time`	✔	✔
`LocalTime`	✔	✔
`DateTime`	✔	✔
`LocalDateTime`	✔	✔
`Duration`	✔	✔
`Point`	✔	✔
`Node`		✔
`Relationship`		✔
`Path`		✔

From Working with Cypher values.

The Java driver maps those types to the following Java types:

Table 2. Mapping from Neo4j / Cypher type to Java
Neo4j type	Java type
`null`	`null`
`List`	`List<Object>`
`Map`	`Map<String, Object>`
`Boolean`	`boolean`
`Integer`	`long`
`Float`	`double`
`String`	`String`
`ByteArray`	`byte[]`
`Date`	`LocalDate`
`Time`	`OffsetTime`
`LocalTime`	`LocalTime`
`DateTime`	`ZonedDateTime`
`LocalDateTime`	`LocalDateTime`
`Duration`	`IsoDuration`*
`Point`	`Point`
`Node`	`Node`
`Relationship`	`Relationship`
`Path`	`Path`

So in any case you get a long out of your database while you are sure you have stored an int or an Integer in the past, you probably have used an embedded instance with a direct connection. Having a value with a smaller width in the database than your application should usually not be a problem, apart from a necessary cast. Having long or double stored and expecting int or float back, you’re risking a loss of precision. The Java driver will throw a LossyCoercion exception.

The preferred way of interacting with Neo4j is through a driver. If you access the embedded database directly, you should make sure to access it in a consistent way. In a scenario where this isn’t possible, for example when you are switching from embedded to cluster, you may have to adapter your client code to use the correct type.

3. Examples

All example code is in the understand-the-type-system project. In TypeConversionTest we set up an embedded instance having the Bolt-Connector enabled as well. See Listing 1.

Listing 1. TypeConversionTest, Setting up an embedded database with Bolt enabled

this.neo4jDb = Files.createTempDirectory("neo4j.db").toFile();

final BoltConnector bolt = new BoltConnector();
this.graphDatabaseService = new GraphDatabaseFactory().newEmbeddedDatabaseBuilder(neo4jDb)
                .setConfig(bolt.type, Connector.ConnectorType.BOLT.name()).setConfig(bolt.enabled, "true")
                .setConfig(bolt.listen_address, LOCAL_BOLT_URL).newGraphDatabase();

this.driver = GraphDatabase.driver("bolt://" + LOCAL_BOLT_URL, AuthTokens.none());

The examples consists of write and read commands. The read commands take an Integer value, store it as a property of a node labeled TestNode and retrieve the nodes id. The write commands take the nodes id and return the value of the property.

We are dealing throughout the examples with this property:

Listing 2. TypeConversionTest, Our test property

static final Integer VALUE_TEST_PROPERTY = 42;

3.1. Write and read directly through the API

Writing looks probably like this:

Listing 3. WriteViaAPICmd, Example of how to write via org.neo4j.graphdb.GraphDatabaseService

try (var transaction = graphDatabaseService.beginTx()) {
        final Node node = graphDatabaseService.createNode(Label.label(NODE_LABEL));
        node.setProperty(NAME_SOURCE_PROPERTY, "written directly via API");
        node.setProperty(NAME_TEST_PROPERTY, value);

        transaction.success();
        return node.getId();
}

A test confirms that reading through the same API returns 42 as Integer again.

Listing 4. TypeConversionTest, Write and read directly through the API

@Test
@DisplayName("Direct API calls should write and read java.lang.Integer as Integer")
public void writingAndReadingViaDirectApi() {
        assertEquals(VALUE_TEST_PROPERTY, new WriteViaAPICmd(graphDatabaseService)
                        .andThen(new ReadViaAPICmd(graphDatabaseService)).apply(VALUE_TEST_PROPERTY),
                        "Returned not the same value as written");
}

Conclusion: The Integer value written can be read back as as Integer again.

3.2. Write and read with Cypher

Given the Cypher statement in Listing 5, we can this use this for writing via the API and the Java-Driver:

Listing 5. TypeConversionTest, Cypher write statement

static final String CYPHER_WRITE = "CREATE (n:TestNode) SET n.numericProperty = $numericProperty, n.source = $source RETURN n";

3.2.1. via the API

Writing with Cypher via embedded API can look like this:

Listing 6. WriteWithCypherViaEmbeddedCmd, Example of how to use Cypher via org.neo4j.graphdb.GraphDatabaseService

final Map<String, Object> parameters = Map.of(NAME_TEST_PROPERTY, value, NAME_SOURCE_PROPERTY,
                "written with Cypher via direct connection to embedded");

try (var transaction = graphDatabaseService.beginTx()) {
        final Long nodeID = graphDatabaseService.execute(CYPHER_WRITE, parameters)
                        .map(row -> ((Node) row.get("n")).getId()).stream().findFirst().get();
        transaction.success();
        return nodeID;
}

Again, the test confirms that reading through the same API returns 42 as Integer:

Listing 7. TypeConversionTest, Write and read with Cypher over embedded connection

@Test
@DisplayName("Cypher over API should write and read java.lang.Integer as Integer")
public void writingAndReadingWithCypherOverEmbeddedConnection() {
        assertEquals(VALUE_TEST_PROPERTY,
                        new WriteWithCypherViaEmbeddedCmd(graphDatabaseService)
                                        .andThen(new ReadWithCypherViaEmbeddedCmd(graphDatabaseService)).apply(VALUE_TEST_PROPERTY),
                        "Returned not the same value as written");
}

3.2.2. via the Java driver

This looks different when using Cypher over the Java driver. That is the point where the Type mapping kicks in.

Listing 8. WriteWithCypherViaBoltCmd, Example of how to use Cypher via the Java driver

final Map<String, Object> parameters = Map.of(NAME_TEST_PROPERTY, value, NAME_SOURCE_PROPERTY,
                "written with Cypher via Java (Bolt) driver");

try (var session = driver.session()) {
        return session.writeTransaction(tx -> tx.run(CYPHER_WRITE, parameters).single().get("n").asNode().id());
}

Now the test looks differen: We have to case the original property to a Long to make it pass:

Listing 9. TypeConversionTest, Write and read with Cypher via the Java driver

@Test
@DisplayName("Cypher over BOLT should write and read java.lang.Integer as Long")
public void writingAndReadingWithCypherOverBoltConnection() {
        assertEquals((Long) VALUE_TEST_PROPERTY.longValue(),
                        new WriteWithCypherViaBoltCmd(driver).andThen(new ReadWithCypherViaBoltCmd(driver)).apply(VALUE_TEST_PROPERTY),
                        "Returned not the same value as written");
}

The driver has converted the Integer value to Long during write and read.

3.2.3. Mixing access patterns

The test in Listing 10 demonstrates mixed access patterns.

Listing 10. TypeConversionTest, Mixing access patterns

@Test
@DisplayName("Mixing different access methods has to be handled with care")
public void mixingDifferentAccessMethods() {
        assertEquals(
                        (Long) VALUE_TEST_PROPERTY.longValue(), new WriteViaAPICmd(graphDatabaseService)
                                        .andThen(new ReadWithCypherViaBoltCmd(driver)).apply(VALUE_TEST_PROPERTY),
                        "Bolt should have used the Java-Driver type mapping");

        assertEquals(
                        (Long) VALUE_TEST_PROPERTY.longValue(), new WriteWithCypherViaEmbeddedCmd(graphDatabaseService)
                                        .andThen(new ReadWithCypherViaBoltCmd(driver)).apply(VALUE_TEST_PROPERTY),
                        "Bolt should have used the Java-Driver type mapping");

        assertThrows(ClassCastException.class, () -> new WriteWithCypherViaBoltCmd(driver)
                        .andThen(new ReadViaAPICmd(graphDatabaseService)).apply(VALUE_TEST_PROPERTY));

        assertThrows(ClassCastException.class, () -> new WriteWithCypherViaBoltCmd(driver)
                        .andThen(new ReadWithCypherViaEmbeddedCmd(graphDatabaseService)).apply(VALUE_TEST_PROPERTY));
}

You’ll be fine if you use Cypher over the Java driver and expect a Long. However, you’ll cannot expect an Integer written through over a driver to return as Integer.

3.3. OGM

OGM will happily map a smaller property stored to a wider attribute as Listing 11 shows, using either a direct connection or a driver connection. Mapping a wider property to to a smaller attribute will fail.

Listing 11. TypeConversionTest, Read with OGM

@Nested
@DisplayName("OGM applies it's own type mapping...")
class OGM {
        @Test
        @DisplayName("...over embedded")
        public void overEmbedded() {

                var sessionFactory = new SessionFactory(new EmbeddedDriver(graphDatabaseService),
                                this.getClass().getPackage().getName());
                final Session session = sessionFactory.openSession();
                assertAll(session.loadAll(TestNode.class).stream()
                                .map(node -> () -> assertEquals(VALUE_TEST_PROPERTY.longValue(), node.getNumericProperty())));
        }

        @Test
        @DisplayName("...over Bolt")
        public void overBolt() {

                var sessionFactory = new SessionFactory(new BoltDriver(driver), this.getClass().getPackage().getName());
                final Session session = sessionFactory.openSession();
                assertAll(session.loadAll(TestNode.class).stream()
                                .map(node -> () -> assertEquals(VALUE_TEST_PROPERTY.longValue(), node.getNumericProperty())));
        }
}