How we put this together

First, we scanned and processed more than 1,000 books, then added the raw text to a database. More books are being added all the time. Explore the library.

Next we combed through the New Testament to identify topic sentences. Our reasoning is that if a person is writing about a spiritual experience, he/she will either quote the Scriptures, or explain it in a way that correlates to the Scriptures. This hypothesis has proven true, and as a result the Scriptures come alive through these myriad stories.

Using the topic sentences, we next used a Sentence Similarity algorithm to find potential matches for each sentence in all 1,000 books. For example, by using the source phrase, “First be reconciled to your brother,” from Matthew 23-24, we found “two brothers who had not spoken civilly to one another for years were among those reconciled,” from Isobel Kuhn’s book Nests Above the Abyss.

Next we did a full text search for the source phrase using the Apache Lucene search engine. The engine returns both exact matches and “fuzzy” matches containing the key words from the phrase, which combine to provide the “fine-toothed comb” for finding stories.

Then we looked at the book database from the other direction. Instead of starting with our topic sentences and searching the texts, we used Topic Clustering algorithms to identify topics and themes we didn’t realize were there. This requires almost two weeks of CPU time!

Finally, we read every matching story for relevance and impact, adding the best to the Scripture Testimony Index.

Each of these steps is described in greater detail in our Data Science section.