Comparing and Assessing Statistical Distance Metrics within the Christian Apostle Paul’s Letters

1 Introduction

This study applies and compares different metrics and statistical methods to Paul’s letters and a variety of roughly contemporary texts also written in Greek. The primary objective is to elucidate the distinctions between several notable metrics and statistical methods of comparison to discern the relative impact of their application on our understanding of Paul’s letters vis-à-vis our comparanda. The secondary objective is to compare the findings from different statistical metrics in order to assess their relative merits as statistical tools for literary analysis. Such analysis extends to determining the statistical significance of different methods’ relative findings and extracting meaningful insights about each metric and method. We aim to determine which metrics are most accurate and provide the most useful information in comparing Paul’s letters to other types of literature such as Greco-Roman philosophy, formal oratory, and so-called Jewish apocalyptic literature. Such findings are especially salient when comparing these quantitative methods to qualitative conclusions from the biblical commentary traditions of literary analysis and comparison.

By comparing computational methods and statistical techniques, this research enhances our understanding of stylometric analysis – a key approach in biblical and literary studies interested in empirical approaches – by suggesting the most effective applications for particular types of data and research questions, thereby yielding more accurate outcomes.^[1] Its interdisciplinary approach bridges computational approaches with traditional biblical and literary scholarship, offering valuable insights into methodological strengths and limitations. To our knowledge, there is no such paper that engages such a mathematical comparison of methods. Appendices include a concise primer on our chosen methods as well as full results from each, for both the methodological novice and those seeking greater detail on our findings.

For these comparative analyses, this article uses the set of data features chosen and explicated in a 2016 monograph, Paul’s Letters and Contemporary Greco-Roman Literature: Theorizing a New Taxonomy.^[2] In that monograph, Robertson makes the case for what are termed “top-down” features, namely features that are mostly second-order categories selected by the analyst trained in this area and hand-coded into Excel spreadsheets. This method is distinct from and contrasted to the so-called “bottom-up” feature selection comprised of vocabulary words and grammatical position, which can be machine-read and require no particular training or expertise in the text or culture in question. The latter (bottom-up) forms the foundation of traditional stylometry and related methods; the former (top-down) is mostly Robertson’s unique contribution to the field. Interested readers can find further discussion on theorizing second-order features in that monograph, where Robertson makes the case that top-down, second-order features both better capture an author’s distinct style and reveal interesting conclusions around comparing Paul’s letters with various contemporary texts.

The feature list is reproduced here, with all subsequent analyses using the hand-coded spreadsheets in Robertson 2016 that indicate the position of every feature occurrence in each text.

Appeals to Authority

Conversation

Prosópopoiia/Éthopoiia

Rhetorical Questions

Metaphors or Analogies

Anecdotes or Examples

Imperatives

Exhortation

Caustic Injunctions

Pathos

Irony or Satire

Hyperbole

Oppositions or Choices

Figurations of Groupness

Plural Inclusive versus Second Person Address

First Person Reflection

Analysis of Potential Questions or Objections

Systematic Argument

2 Methods Introduction

This section introduces each method, describing its function, relevant mathematical information, typical applications, and utility for our research questions, including strengths and limitations.

2.1 Euclidean Distance

A key method in author attribution, Euclidean distance aids stylometry by quantifying stylistic differences between texts and revealing textual patterns.^[3] Euclidean distance calculates the separation between texts in a high-dimensional space, considering attributes like word frequency and sentence structure, both key stylometric criteria. Euclidean distance is defined by the following:

d ( p , q ) = ∑ i = 1 n ( q i − p i ) 2 .

A simple example to illustrate: imagine you are standing at one corner of a room, and you want to get to the opposite corner. The Euclidean distance is the straight line that takes you directly to that corner. In a two-dimensional space, the formula to compute this straight-line distance between two points (say, Point 1(x ₁, y ₁) and Point 2(x ₂, y ₂)) is:

( ( x 2 − x 1 ) 2 + ( y 2 − y 1 ) 2 ) ,

the Euclidean distance between two points P1(x ₁, y ₁) and P2(x ₂, y ₂) in a 3D space is given by the following formula:

d ( P 1 , P 2 ) = ( ( x 2 − x 1 ) 2 + ( y 2 − y 1 ) 2 + ( z 2 − z 1 ) 2 ) .

So, the distance will be calculated as the square root of the sum of the squares of differences in both x and y coordinates.

In the context of stylometric analysis, Euclidean distance serves as a crucial tool for measuring the similarity or dissimilarity between texts based on their stylometric features.

Euclidean space is a geometric construct that provides a framework for understanding the relationship between texts in terms of their stylometric attributes. Stylometric features encompass various linguistic attributes such as word frequency, length, syntactic structures, and vocabulary richness. Each text can be represented as a point in a high-dimensional space, with each coordinate corresponding to the values of these stylometric features. Euclidean distance calculates the separation between these points, indicating the similarity or dissimilarity between texts.

Beyond stylometric analysis, Euclidean distance finds applications in various fields, including machine learning, data science, computer vision, image processing, GIS, and spatial analytics. It is used in algorithms like K-Nearest Neighbors, K-Means, and Principal Component Analysis for classification, clustering, and dimensionality reduction tasks. In computer vision, Euclidean distance aids in pattern recognition, similarity checks, and object detection. In GIS, it helps calculate geographical distances and perform spatial operations.

One of the most significant advantages of Euclidean distance is its simplicity and intuitive calculation method. It can be applied across different dimensions, making it highly versatile for multi-dimensional data analysis tasks, such as assessing the proximity of texts that are coded according to many different features. Additionally, it represents the most direct path between two points, facilitating easy interpretation and understanding.

Euclidean distance has limitations in stylometric analysis and other applications. One of its key limitations is that it is sensitive to scale. This means that the distances computed might be skewed depending on the units of the features. Another limitation is that as the dimensionality of the data increases, Euclidean distance tends to become less effective – a phenomenon known as the “curse of dimensionality.” Euclidean distance also does not work well when the data are sparse. The vectors should be non-zero in approximately 75% of attributes for Euclidean distance to be usable, a limitation that applies to any other norm-induced distance measures – such as Manhattan distance, discussed below. In textual data, other distance measures are typically preferred over Euclidean distance, because Euclidean distance depends on the vector’s magnitude instead of the angle between the vectors. The angle measure is more resilient to variations of occurrence counts between semantically similar terms, whereas occurrence counts and heterogeneity of word neighborhoods influence the magnitude of vectors. Even if the two text documents are far apart by Euclidean distance, in other words, there is still a chance that they are close to each other in terms of their context.

2.2 Cosine Similarity

Cosine similarity is a pivotal concept in mathematics and computational linguistics, measuring the similarity between vectors by calculating the cosine of the angle between them, commonly used in stylometry for comparing textual documents represented as Bag-of-Words (BoW) or Term Frequency-Inverse Document Frequency (TF-IDF) representations.^[4] These representations transform textual documents into numerical vectors, where each element corresponds to the frequency or importance of a specific term within the document. By employing such representations, complex textual information is condensed into structured numerical formats, facilitating computational analysis and comparison. Cosine similarity operates by calculating the cosine of the angle between two non-zero vectors. Mathematically, it is expressed as the dot product of the vectors divided by the product of their magnitudes. This formulation enables the comparison of documents based on their directional similarity rather than their magnitudes, making it robust to variations in document lengths and word frequencies.

This similarity metric, ranging from −1 to 1, determines perfect alignment at 1, perfect dissimilarity at −1, and orthogonality at 0, making it robust for tasks like document clustering, authorship attribution, plagiarism detection, and information retrieval in natural language processing. In collaborative filtering-based recommendation systems, cosine similarity helps identify users or items having similar preferences. This technique is commonly used in movie, music, and product recommendations. Cosine similarity can also be used in text classification tasks such as spam detection, sentiment analysis, or topic modeling. Search engines use cosine similarity to match user queries with relevant documents by comparing the query vector to document vectors. This approach is commonly employed in information retrieval. Finally, cosine similarity is useful for clustering similar data points together, for example to group similar news articles or social media posts.

Cosine similarity is defined by the following:

Similarity ( A , B ) = cos ( θ ) = A · B | | A | | | | B | | = ∑ i = 1 n A i B i ∑ i = 1 n A i 2 ∑ i = 1 n B i 2 .

Below is a simple example of cosine similarity:

Cosine similarity is a fundamental concept in mathematics and computational linguistics, offering a powerful tool for measuring the similarity between two vectors in an inner product space. In the context of stylometry, which concerns the quantitative analysis of literary style, cosine similarity serves as a cornerstone for comparing textual documents based on their vectorized representations.

Cosine similarity is easy to compute, especially with sparse matrices, and it can capture the overall similarity of the documents regardless of their length. Furthermore, cosine similarity is remarkably versatile. It can be applied not only to pairs of points but also to a diverse array of data types, including text documents, images, and numerical data. This versatility renders cosine similarity a powerful tool in various domains. However, it lacks consideration for the magnitude of vectors, focusing solely on their direction – it yields the same value regardless of the size of the vectors being compared as long as the angle between them is the same. This can be problematic when comparing vectors of different sizes, such as when comparing the risk profiles of different companies. It also does not capture the semantic meaning of words, making it less effective in understanding context. Additionally, in high-dimensional spaces where data are sparse, cosine similarity can be less reliable – it might not accurately reflect the true similarity between data points. Another limitation is that cosine similarity depends on vector normalization. Different normalization methods can yield different results, making comparisons sensitive to preprocessing choices.^[5]

2.3 Jaccard Similarity

Jaccard similarity is a key tool in data analysis and information retrieval, quantifying the similarity between sets by comparing their overlap.^[6] It calculates the ratio of intersecting elements to the total elements in the sets, providing a score from 0 to 1 where 1 signifies perfect similarity, and 0 indicates no common elements. It is defined by the following equation:

J ( A , B ) = | A ∩ B | | A ∪ B | .

Here is a simple example of Jaccard similarity:

To calculate Jaccard similarity, the number of common elements (intersection) between two sets is divided by the total number of distinct elements (union) present in both sets. This ratio ranges from 0 to 1, where 1 signifies perfect overlap and 0 indicates no common elements.

Jaccard similarity is a fundamental concept in data analysis and information retrieval, offering a straightforward method to quantify the similarity between sets. Its applications span various domains, including text analysis, genomic analysis, social network analysis, and data deduplication. Jaccard similarity offers a different perspective in stylometric analysis, enabling the comparison of text documents based on their word content. By representing texts as sets of words and computing the ratio of intersection to union, analysts can discern similarities and differences in writing styles.

Jaccard similarity is a useful metric that has applications in various fields. One of its popular applications is in Natural Language Processing (NLP), where it is used to compare the similarity of two text documents. This technique is used in document clustering, plagiarism detection, and recommendation systems. Jaccard similarity is also used in recommendation systems based on collaborative filtering. It helps identify users with similar preferences by comparing the sets of items each user has interacted with. This way, recommendations can be made based on the preferences of similar users. In the field of genomic analysis, biologists use Jaccard similarity to measure the similarity between genetic sequences. By identifying common elements and mutations, Jaccard similarity aids in understanding evolutionary relationships. Social network analysis similarly uses Jaccard similarity to help analyze social networks by comparing the sets of friends or connections between individuals, thereby more easily identifying users with similar social circles. Finally, Jaccard similarity can also be used in data cleaning and preprocessing tasks. It can detect and remove duplicate records by comparing the sets of attributes within records to help in data deduplication, which is essential in maintaining data quality.

This metric is highly recommended for comparing sets of data for more conceptual reasons. One of the most significant advantages of using Jaccard similarity is that it is scale-invariant, which means that the size of the sets being compared does not affect the metric’s performance. This feature makes it a robust metric for comparing sets of varying sizes. Another advantage of Jaccard similarity is that it is easy to understand. Its simple concept makes it accessible to both data scientists and non-technical stakeholders. Finally, Jaccard similarity is versatile and can be applied to a wide range of data types and domains. This makes it a highly flexible metric for similarity measurement, from text data to biological data.

However, Jaccard similarity has certain limitations. First, it treats all elements equally without considering their frequency of occurrence. This means that words appearing multiple times in a text receive the same weight as words appearing only once, which can be a limiting factor as word frequency often carries important stylistic and authorial information. Second, Jaccard similarity is sensitive to the size of the sets being compared. Even slight variations in the size of the sets can lead to significant changes in the similarity score, which may not always accurately reflect the similarity between texts. Third, Jaccard similarity only considers the presence or absence of elements, completely ignoring their order or arrangement within the sets. This can be problematic in stylometric analysis, where the sequential arrangement of words, phrases, or stylistic features may carry important information about an author’s style. Finally, Jaccard similarity is based on set operations and does not capture semantic relationships between words, which is a significant limitation. Due to this, it may not effectively capture similarities between texts that use synonyms, paraphrasing, or related concepts.

2.4 Manhattan Distance

Manhattan distance is a practical measure in mathematics and computer science that is useful in grid-based systems where movement is constrained to horizontal and vertical paths. Stylometric analysis quantifies textual similarities by summing the absolute differences between corresponding linguistic features, aiding tasks like authorship attribution and route planning across various fields. It is defined by the following:

Manhattan distance = ∑ i = 1 n ∣ x i − y i ∣ .

A simple example of Manhattan distance:

In stylometric analysis, Manhattan distance can be a valuable tool. By representing texts as vectors, with each element corresponding to the frequency of certain linguistic features (such as word occurrences, character n-grams, or syntactic structures), analysts can compute the Manhattan distance between these vectors to quantify the dissimilarity or similarity between texts.

Manhattan distance provides a straightforward and versatile method for measuring distance within grid-based systems, and its application extends beyond mathematical contexts to fields such as stylometry, where it facilitates the quantitative analysis of textual similarities and differences. Once the Pauline letters are represented as vectors in the feature space, the Manhattan distance can be calculated between pairs of letters. For each pair of letters attributed to different authors (e.g., Paul and another potential author), the Manhattan distance is computed by summing the absolute differences between the corresponding components of their feature vectors. Moreover, Manhattan distance is advantageous in scenarios where the data are naturally discrete or when the cost of movement along different dimensions is not uniform. This makes it a preferred choice in many practical applications, including route planning, image recognition, and clustering algorithms.

Manhattan distance is less affected by outliers compared to Euclidean distance. It assigns equal weight to each dimension and is more robust to variations in individual coordinates. If your data have outliers or you want to reduce the impact of extreme values, Manhattan distance might be a better choice. Also, you might want to consider the Manhattan distance if the input variables are not similar in type (such as age, gender, height, etc.). Due to the curse of dimensionality, we know that Euclidean distance becomes a poor choice as the number of dimensions increases. As it does not take any squares, it does not amplify the differences between any of the features. It also does not ignore any features.

Manhattan distance is a distance metric that treats each dimension or coordinate equally, regardless of the scale or importance of the feature. However, in cases where certain dimensions are more significant than others, this can lead to inaccurate distance measurements. Additionally, Manhattan distance restricts movement to horizontal and vertical paths, which can be inadequate for situations where diagonal movement is allowed or relevant. In such cases, other distance metrics like Euclidean distance might provide a better representation of true distances. It’s worth noting that while Manhattan distance excels in grid-based systems like city maps or game boards, it might not be the most appropriate metric for continuous or non-grid systems where movement is not constrained to specific paths.

3 Data Analysis: Analyzing Paul’s Letters Using Multiple Methods

In this section, we outline the steps taken to compute and analyze the similarities and differences among various texts using the distance and similarity measures above: Euclidean distance, cosine similarity, Manhattan distance, Jaccard similarity, and Burrows’ Delta Method.

First, we defined the feature vectors for each text. These vectors represented the frequency of specific textual features. For instance, the feature vectors for texts like “1 Corinthians,” “2 Corinthians,” and others were compiled into a dictionary, allowing for structured and systematic analysis. Here is an example of the feature vectors defined for each text:

We then created DataFrames from this dictionary to facilitate computational analysis. Using the DataFrames, we calculated the Euclidean distance, cosine similarity, and Manhattan distance. The calculation of these metrics involved the following code:

The Jaccard similarity was calculated using a custom function designed to handle numerical arrays:

The code for Burrows’ Delta Method involved several steps, including calculating z-scores and delta scores:

In this code, several Python libraries were used to allow for easy and accurate calculations.^{[9] , [10]} The Pandas library was important for data manipulation and analysis, providing helpful data structures like DataFrame, which are essential for handling large datasets. NumPy was another library used in our code, supporting large, multi-dimensional arrays and matrices. It offers a collection of mathematical functions to operate on these arrays, which was needed for performing numerical operations and calculations necessary for the analysis. SciKit-Learn, a machine learning library, was used extensively to calculate the Euclidean distance, cosine similarity, and Manhattan distance. It offers simple and efficient tools, enabling ease of application of the various distance and similarity measures. Finally, the OS module, a standard utility in Python, was used to interact with the operating system. It aided with reading from and writing to files, ensuring smooth handling of input and output operations.

4 Discussion of Euclidean Distance Results

In analyzing the Euclidean distances between the various texts, we can draw several important conclusions about their relationships and differences. In our context, each text is represented as a point in a 20-dimensional space where each dimension corresponds to the frequency of a specific rhetorical feature – 20 features in all. By examining these distances, we can infer the degree of similarity or difference between the texts based on their rhetorical profiles.

The numerical range of Euclidean distances can vary significantly. We identify the maximum and minimum distances observed to set a context for what counts as high or low (Figure 1).

Figure 1

Euclidean distances.

Low distance pairs such as 1 Corinthians and Galatians, with a distance of 64.76, indicate a close relationship, suggesting that these texts share similar patterns and frequencies for the comparanda. Similarly, 2 Corinthians and 1 Thessalonians, with a distance of 30.54, show a high degree of similarity, indicating that their usage of the comparanda is quite alike.

Moderate distance pairs like Romans and 1 Corinthians, with a distance of 42.92, and Hebrews and 1 Corinthians, with a distance of 45.19, suggest some degree of similarity but also notable differences. These moderate distances imply that while there are commonalities between these texts, they are not as closely related as the low-distance pairs.

High-distance pairs, such as Epictetus’ Discourses, consistently show high distances from all other sets, with the smallest distance being 522.21 to 1 Corinthians. This indicates that Epictetus’ Discourses are quite distinct from the other texts in terms of the frequency and pattern of the comparanda. Additionally, Philemon and 1 Corinthians, with a distance of 89.32, suggest significant differences, making them less comparable.

Interpreting these distances, low numbers (0–30) indicate strong similarities between the sets, suggesting that the compared documents share a high frequency of similar comparanda. Moderate numbers (30–60) suggest a moderate level of similarity, implying that while there are commonalities, there are also notable differences. High numbers (60 and above) indicate significant dissimilarities. For instance, values above 500 suggest that the documents are quite different in terms of the chosen comparanda, which might reflect different authors, genres, or thematic focuses.

5 Discussion of Manhattan Distance Results

In the context of rhetorical feature analysis, Manhattan distance provides insights into the total difference in feature counts between texts. Unlike Euclidean distance, which considers the shortest path in a multi-dimensional space, Manhattan distance strictly follows the axes, offering a different perspective on textual similarity.^[11]

Manhattan distances can range from 0 to a large number, depending on the feature counts. Lower values indicate greater similarity, while higher values denote greater dissimilarity. For practical analysis, distances closer to 0 can be considered indicative of high similarity, while larger values reflect more substantial differences (Figure 2).

Figure 2

Manhattan distances.

The low Manhattan distance of 98 between 1 Corinthians and Galatians suggests that these two texts have very similar frequency distributions of the selected comparanda, indicating a close thematic and rhetorical resemblance. This implies that these letters may share similar contexts or purposes, potentially reflecting common authorship or similar influences in their composition. Similarly, the distance of 110 between Romans and Galatians also reflects a high degree of similarity, supporting the notion that these letters are closely related in their use of rhetorical and thematic elements.

In the moderate distance category, the distance of 165 between 2 Corinthians and 1 Thessalonians suggests some differences in the frequency of comparanda, indicating that while there are shared themes, there are also divergent elements that differentiate these texts. This could reflect variations in the contexts or audiences addressed by these letters. The distance of 180 between Philippians and 1 Thessalonians further supports this, hinting at some shared rhetorical strategies but also distinct variations in content and style.

High Manhattan distances, such as the 1,236 between Epictetus’ Discourses and 2 Thessalonians, indicate significant differences in the frequency of comparanda. This suggests that these texts have distinct themes and rhetorical strategies, likely reflecting different genres, purposes, or authorship. Similarly, the high distance of 934 between Philodemus’ On Death and Epictetus’ Discourses highlights substantial differences in content and style, reinforcing the idea that these texts are markedly distinct in their rhetorical and thematic approaches.

The interpretation of these distance scores helps in understanding the degree of similarity and difference between the texts. Low distances (below 200) indicate high similarity in the frequency distributions of comparanda, suggesting thematic and rhetorical consistency. Moderate distances (200–600) suggest some similarities but also notable differences, reflecting a mix of shared and distinct elements. High distances (above 600) highlight significant differences in the frequency distributions, pointing to distinct themes, topics, and rhetorical strategies.

6 Discussion of Cosine Similarity Results

In analyzing the cosine similarity between the various texts, we can determine the degree of alignment or similarity in their rhetorical features. Cosine similarity measures the cosine of the angle between two vectors, which in this context are the rhetorical feature frequency vectors of the texts. Cosine similarity values range from −1 to 1, though in the context of text analysis, values typically range from 0 to 1. A similarity close to 1 indicates high similarity, while a value near 0 indicates low similarity. For example, values above 0.8 could be considered high, indicating strong similarity, whereas values below 0.3 could be considered low, indicating significant differences (Figure 3).

Figure 3

Cosine similarity scores.

High similarity pairs such as 1 Corinthians and Galatians, with a cosine similarity score of 0.945160, suggest a strong thematic and rhetorical congruence between these texts. This high score indicates that the frequency of the chosen comparanda in these letters is quite similar, pointing toward a consistently shared conceptual focus and rhetorical style. Similarly, the high similarity score of 0.900998 between Romans and Hebrews highlights a strong resemblance in the usage of comparanda, suggesting shared theological themes and rhetorical strategies, possibly indicative of similar authorship or common theological discourse. Another pair exhibiting high similarity is 1 Corinthians and Romans, with a score of 0.912127. This reinforces the idea that Paul’s letters share significant thematic and stylistic consistency. Such high similarity scores suggest that these letters might have been intended for similar audiences or contexts, emphasizing consistent theological teachings and rhetorical methods.

On the other hand, moderate similarity pairs such as 2 Corinthians and 1 Thessalonians, with a score of 0.782958, indicate some divergence in themes or rhetorical approaches while still maintaining notable similarities. This score suggests that while these letters share some common elements, there are also distinct differences possibly arising from different contexts or shifts in the author’s focus. Similarly, Philippians and Galatians, with a score of 0.724144, exhibit moderate similarity, suggesting shared rhetorical elements and theological points but also highlighting variations that could stem from different situational contexts.

Low similarity pairs, such as Philodemus’ On Death and 2 Thessalonians, with a score of 0.097214, indicate significant differences in the frequency of comparanda between these texts. This low score suggests that the rhetorical styles and themes in these works are very different, reflecting distinct genres or thematic focuses. Similarly, the low similarity score of 0.334328 between Epictetus’ Discourses and 2 Thessalonians perhaps underscores the contrast between the philosophical nature of Epictetus’ work versus the more religious content of 2 Thessalonians.

In interpreting these similarity scores, high similarity (0.8–1) indicates a high degree of thematic and rhetorical consistency, suggesting common authorship, similar audience, or consistent theological focus. Moderate similarity (0.5–0.8) suggests shared elements with notable differences, possibly arising from different writing contexts or shifts in rhetorical strategy. Low similarity (0–0.5) highlights significant differences in content and style, indicating distinct themes, purposes, or rhetorical techniques, and potentially different authors. It is important to note that there is no universally defined threshold for determining what constitutes a low or high cosine similarity score. The classification of a score as high or low is contingent upon the specific data being analyzed and is derived from internal comparison to other scores instead of an objective bar for significance.

7 Discussion of Jaccard Similarity Results

The Jaccard similarity index provides a different perspective on the relationship between texts by focusing on the presence or absence of specific features rather than their frequency. In this context, the sets are the rhetorical features present in each text. Jaccard similarity ranges from 0 to 1, where 0 indicates no common features and 1 indicates identical feature sets (Figure 4).

Figure 4

Jaccard similarity scores.

High similarity pairs include 1 Corinthians and Galatians with a Jaccard similarity score of 0.579618. This relatively high score indicates a significant overlap in the presence of comparanda between these two letters, suggesting that similar themes and rhetorical elements are prominent in both. Another high similarity pair is 4 Maccabees and Galatians, which have a Jaccard similarity score of 0.571429. This high similarity points to common rhetorical or thematic elements in these texts, despite their different contexts.

Moderate similarity pairs include 2 Corinthians and 1 Thessalonians, which have a Jaccard similarity score of 0.378571. This score suggests a moderate level of overlap in the comparanda used, indicating some shared themes or rhetorical strategies but also notable differences. Another moderate similarity pair is Philippians and 1 Thessalonians, with a Jaccard similarity score of 0.513158. This score reflects a moderate degree of overlap, hinting at shared but not identical themes or rhetorical approaches.

Low similarity pairs include Philodemus’ On Death and Epictetus’ Discourses, with a Jaccard similarity score of 0.019305. This very low similarity indicates minimal overlap in the comparanda, suggesting significant differences in content and style. Another low similarity pair is Philodemus’ On Death and 2 Thessalonians, which have a Jaccard similarity score of 0.148649. Similarly, a low similarity score highlights substantial differences between these texts.

High similarity scores (0.4–0.6) indicate a significant overlap in the presence of comparanda, suggesting shared themes, topics, or rhetorical strategies. Moderate similarity scores (0.2–0.4) suggest a moderate degree of overlap, indicating some common elements but also substantial differences. Low similarity scores (0–0.2) highlight minimal overlap, pointing to distinct differences in themes, topics, and rhetorical strategies.

8 Discussion of Burrows’ Delta Method Results

In Burrows’ Delta method, lower scores indicate greater stylistic similarity, while higher scores suggest more substantial stylistic differences. Each cell in the matrix represents the delta score between two texts (Figure 5).

Figure 5

Burrows’ delta method scores.

The delta score between 1 Corinthians and Galatians is 0.512, indicating a high degree of stylistic similarity. This suggests that the two texts share many stylistic features, possibly reflecting similar authorship or a close temporal or contextual relationship. The score between Hebrews and 1 Corinthians is 0.676, also suggesting a significant stylistic similarity. These low delta scores, ranging from 0.512 to 0.800, suggest that the texts compared have a high degree of stylistic similarity. For instance, texts like 1 Corinthians and Galatians (0.512) and 1 Corinthians and Hebrews (0.676) might have been written by the same author or in a similar style.

In the realm of moderate similarity pairs, the delta score between 1 Thessalonians and 2 Corinthians is 0.837, indicating a moderate stylistic difference. This suggests that while there are some differences in stylistic features, the two texts are still relatively similar compared to others. The score between Romans and Galatians is 0.656, showing moderate similarity, which might reflect shared authorship or thematic connections. These medium delta scores, ranging from 0.800 to 1.100, indicate moderate stylistic differences. Texts in this range, such as 2 Thessalonians and Romans (0.855), while not identical in style, share some common stylistic features. This might indicate that different authors are writing in a similar style or that the same author is writing in different contexts.

Low similarity pairs include Epictetus’ Discourses and 1 Thessalonians, with a delta score of 1.303, indicating substantial stylistic differences. This is expected as these texts likely belong to different genres and authors. Similarly, the score of 1.084 between Aelius Aristides’ Panathenaicus and 1 Corinthians reflects significant stylistic differences, which could be attributed to different authorships as well as the highly different contexts and social purposes. High delta scores, ranging from 1.100 and above, reflect significant stylistic differences. Examples such as Epictetus’ Discourses and 1 Thessalonians (1.303) or Aristides’ Panathenaicus and 1 Corinthians (1.084) suggest different authors, genres, or contexts, highlighting distinct stylistic features.

9 Notable Similarities and Divergences

The visualizations and detailed analysis of textual similarities and divergences across different metrics provide a comprehensive understanding of the relationships between the texts. By examining the top five and bottom five similarity and distance scores, we can draw several notable insights and conclusions.

Graph of the top five pairs of texts that are most similar according to each method.

Graph of the bottom five pairs of texts that are least similar according to each method.

The high similarity between 1 Corinthians and Galatians is evident across multiple metrics, including cosine similarity, Jaccard similarity, and Burrows’ Delta Method. This indicates that these texts share a significant number of common features and have a similar distribution of these features. For instance, the cosine similarity score between these texts is among the highest, reflecting their close alignment in terms of feature vectors. Similarly, the Jaccard similarity score shows a substantial overlap in features, while the Burrows’ Delta score indicates minimal stylistic differences. The consistent high similarity scores across different metrics suggest that 1 Corinthians and Galatians are thematically and stylistically close. This is consistent with the known historical and contextual connections between these letters, both written by Paul addressing similar issues in early Christian communities.

Conversely, Epictetus’ Discourses consistently shows high distances and low similarity scores with other texts across all metrics. For example, it has a high Euclidean distance of 522.21 with 1 Corinthians and a low Jaccard similarity of 0.061 with 2 Corinthians. This indicates that Epictetus’ Discourses is markedly different from the other texts in terms of both feature counts and distributions, as well as stylistic elements. The consistent identification of Epictetus’ Discourses as an outlier across multiple metrics highlights its unique characteristics and suggests that it may belong to a different genre or context compared to the other texts analyzed.^[12]

The analysis also reveals that 1 Corinthians and 2 Corinthians have a close similarity across various metrics. For instance, their Euclidean distance is relatively small, indicating that the overall magnitude of differences in feature counts between these texts is minimal. Their high cosine similarity score further supports this, showing that the direction of their feature vectors is closely aligned. Additionally, the Jaccard similarity score indicates a substantial overlap in features, reflecting the shared themes and stylistic elements. This high degree of similarity is expected given that both letters were written by Paul to the same community, addressing related issues.

Another notable finding is the similarity between 1 Thessalonians and Philippians. These texts show high similarity across metrics, with particularly high Manhattan and Euclidean distance scores, suggesting minimal differences in feature magnitudes. The Jaccard similarity score highlights a significant overlap in features, consistent with the expectation that these letters, written by the same author to the same community, would be thematically and stylistically similar. Furthermore, Romans and Hebrews show a high similarity score across cosine similarity, Jaccard similarity, and Burrows’ Delta Method, indicating a significant overlap in themes and stylistic features. The Euclidean and Manhattan distances between these texts are moderate, suggesting that while there are differences in feature magnitudes, the overall distribution patterns are similar. This relationship may reflect shared theological themes or stylistic influences between these texts.

On the other hand, Philemon shows lower similarity scores and higher distance measures with most other texts. For instance, its Jaccard similarity with 1 Corinthians is very low, reflecting fewer shared features. The Euclidean and Manhattan distances are also relatively high, indicating more substantial differences in feature counts. This suggests that Philemon may have a unique set of features, or a different style compared to the other texts analyzed.

The bottom five similarity scores highlight texts with minimal overlap or significant differences. For example, the low similarity between Philemon and Seneca’s Natural Questions across multiple metrics underscores the distinct thematic and stylistic elements of these texts. Similarly, the high Euclidean and Manhattan distances between Epictetus’ Discourses and other texts further emphasize its unique characteristics and outlier status.

Triangulation, where three or more methods indicate high similarity between the same pairs of texts, further validates these findings. For instance, the high similarity between 1 Corinthians and Galatians is supported by cosine similarity, Jaccard similarity, and Burrows’ Delta Method, providing robust evidence of their close relationship. Similarly, the outlier status of Epictetus’ Discourses is consistently highlighted across all metrics, reinforcing its distinctiveness.

The visualizations illustrate these patterns clearly. The top five similarity scores graph shows texts like 1 Thessalonians with high similarity across multiple metrics, indicating its close relationship with multiple other texts. In contrast, the graph of the bottom five similarity scores highlights texts like Epictetus’ Discourses and Philemon, which show significant differences from others. The graphs are plotted on a log scale to effectively represent the wide range of similarity and distance scores, allowing for a clearer visualization of both small and large values. Using a log scale helps to compress the range, making it easier to compare values that vary by order of magnitude. This approach is particularly useful in text analysis, where the similarity and distance measures can differ significantly between text pairs. For those interested, Appendix B includes all pairs, providing a complete overview of the relationships among the analyzed texts.

10 Conclusion

The comprehensive analysis of Paul’s letters using multiple similarity and distance measures, including Euclidean distance, cosine similarity, Manhattan distance, Jaccard similarity, and Burrows’ Delta Method, reveals significant insights into the textual and stylistic characteristics of these epistles. This study underscores the importance of employing diverse analytical methods to capture the nuanced relationships between texts. Each metric highlighted different aspects of similarity and dissimilarity, which can provide a multi-faceted understanding of the letters attributed to Paul.

The application of Euclidean distance provided a measure of overall dissimilarity where magnitude matters, revealing significant differences between texts such as Epictetus’ Discourses and others, emphasizing the unique feature distribution of this text. Cosine similarity, which is ideal for documents with varying lengths, showed high similarity between 1 Corinthians and Galatians, suggesting strong thematic and rhetorical consistency. This consistency was further evidenced by high Jaccard similarity scores, indicating significant overlap in the presence of comparanda, such as between 1 Corinthians and Hebrews. Manhattan distance, known for its robustness to outliers, highlighted consistent feature count differences, particularly between 1 Corinthians and Philemon, suggesting divergent themes or rhetorical approaches. Burrows’ Delta Method, focusing on stylistic features, showed high similarity scores between Romans and Hebrews, indicating shared authorship or thematic connections.

These findings can usefully deeper insights into the Pauline corpus beyond what traditional qualitative analysis can offer. By quantitatively measuring the similarities and differences among Paul’s letters, we can make more informed assertions about their authorship, thematic coherence, and rhetorical strategies. For instance, the consistently high similarity scores between certain letters reinforce the argument for their inclusion in Paul’s authentic corpus. In contrast, others, like 2 Thessalonians, show significant stylistic differences, supporting the hypothesis of different authorship.

The implications of these findings extend to biblical scholarship, textual criticism, and historical theology. They offer a robust framework for analyzing ancient texts, combining computational methods with traditional literary analysis. This dual approach not only enhances our understanding of Paul’s letters but also provides a model for studying other corpora with similar complexity. Furthermore, putting these methods side by side allows us to triangulate findings, providing a more reliable and comprehensive view of the texts. The integration of various metrics reveals patterns and relationships that might be overlooked when using a single method. For example, while Euclidean distance highlights overall dissimilarities, cosine similarity, and Jaccard similarity focus on thematic and pattern overlaps, offering a balanced perspective. Ultimately, this study illustrates the value of combining quantitative and qualitative approaches in textual analysis. It shows that by leveraging multiple methods, researchers can uncover richer, more nuanced insights into ancient texts, enhancing our understanding of their historical and literary contexts.