Patterns of Everyday Exchange: Big Historical Data and the Case of the Basel Advertisement Paper, 1729–1844

1 The Source, the Data and the Problem of Classification

The Avisblatt (literally “announcement sheet”) of Basel first appeared on January 4, 1729, making it the first Swiss Intelligenzblatt or advertisement paper. It was modelled after the advertisement paper of Frankfurt that had started seven years earlier (there are also a few French and British precursors in the 17^th century). The Avisblatt, in turn, immediately became a model for similar papers in Zurich, Bern, and other Swiss cities. Other than the Prussian intelligencers, the Avisblatt was a privately owned and run paper, operated under public privilege. It was edited first by Johannes Burckhardt, then his son-in-law Peter Raillard and his descendants – a family media business run over four generations. The Avisblatt was published weekly, since the late 1830s increasingly more often.

All 6,626 issues published during the 116 full years in which the Avisblatt was a pure advertisement paper (1729-1844) are preserved. In the SNSF research project “Printed markets”, we have digitized and processed a full set. The University Library in Basel produced and co-financed high-quality digital facsimiles, the technical backbone of the resulting digital collection is an iiif-based annotation infrastructure called Freizo, developed by the not-for-profit company Data Futures GmbH. We imported the digital facsimiles into the Transkribus environment, where each of the roughly 41,000 pages of advertisements was segmented, i.e., each section header, each advert, and each notice was defined as a separate text region. The tool dhSegment was used to predict those text regions, which were then corrected and refined by hand. For the following text recognition, we trained two models in Transkribus with a Character Error Rate (CER) in the validation set of less than 1.7 percent.

The data was exported from Transkribus to three different venues: It is intended that the digital version of the Avisblatt becomes provided and preserved both in the newspaper repository of the Swiss National Library (e-newspaper-archives.ch), and in the form of an InvenioRDM repository on the hasdai platform provided by Data Futures GmbH. Thirdly, utilizing R and Github as our data science environment, we also processed the structured texts (without the images) into a database of 847,000 single advertisements and 85,000 notices about marriages, deaths, elections etc. Finally, we enriched the data with layers of metadata, mostly by running R scripts tailored to the task. One important aspect, for example, was to identify whether a specific advert is a reprint of an advertisement posted in the previous issue, or if it is an original posting: by default, advertisements were reprinted once, unless the advertiser notified the Avisblatt editor that the posting already served its purpose. Overall, 79 percent of the 450,000 original postings became reprinted.

As the advertisements are both extremely numerous and diverse in nature, the data – a giant and relatively unstructured heap of texts – need to be reduced into a manageable form: may it be to extracting an interesting, relevant subset of ads for further inspection and analysis (e.g., housing ads), or for aggregating information by summary statistics. For both cases, advertisements need to be classified. The internal classification of the source (it’s mostly “classified ads”, after all) is helpful, but neither detailed, nor consistent in class type: The sections “for sale”, “sought to buy”, “to be lend”, “sought to be lend” distinguish buying and lending as well as supply and demand, but they offer no clues as to what type of good or service is advertised. The sections “Lost and found” and “Boarding, services and information offered and requested” are topically somewhat more specific, but do not allow to distinguish offers from requests. The section “All sorts of notices”, finally, is a very mixed bag that does neither.

At the outset of the project, it was clear that for sheer volume, the ads could not all be manually classified by the team in a reasonable amount of time. More detailed classification information was added manually for four sample years only – still 13,085 ads, though – using a classification system with 37 labels. ^[10] This sample can be used as a ground truth to check other classification methods against, and it can also serve as the starting point for the established algorithmic method to classify texts, supervised machine learning (SML).

In SML, sample data is used to predict if an unclassified text is more likely to belong to a specific class or not. A classifier can be judged by two indicators, its precision (which share of the texts predicted to be in the class actually belong there) and sensitivity (which share of the ads that belong to a class are actually predicted to be in that class). We tested two different SML methods, using three quarters of the manually classified ads (randomly drawn) as a test set and the remaining part as a validation set, to gauge precision and sensitivity. As ads are generally a rather short type of text, it needs a large sample to get a good classifier for a specific type of advertisement. It turned out that to reach 80 percent both for precision and sensitivity, one usually needs to identify a sample of at least 2,500 ads that are of the pertinent type – for some interesting, more specific types of ads, there are not even that much in total. Thus practically, only very large classes of Avisblatt ads could be classified with supervised machine learning (given our specific manual classification, essentially only housing and lost & found).

We decided to develop a different approach of algorithmic tagging, a dictionary-based approach we refer to as “dynamic tagging”. Here, the investment that must be made for each classifier is not tagging thousands of ads manually, but thinking of, testing, and refining vocabulary that is distinctive for that class of ads. In our Avisblatt R package, we define a class of functions (“tag filter”) that operate as lexical classifiers. Each contains two dictionaries of regular expressions (i.e., sequences of characters that specify a search pattern), a positive dictionary to match ads that we want to include, and a negative dictionary to exclude unwanted by-catch. Each function can be restricted to ads that appear under certain headers in the source; single ads can also be included or excluded by adding them to a list included in the tag filter. Due to the script-based approach of R, we can update the metadata anytime we add or change tag filters, so we employ a dynamic instead of a static classification approach.

As with SML classification, dictionary-based classification necessarily produces false positives and false negatives, it tags ads that are in fact not pertinent to the specific topic (lack of precision) and misses out on tagging some that really are (lack of sensitivity). A main task in developing a new tag filter is expanding and adjusting the dictionaries in such a way as to make the filter more selective and precise. In SML classification, this is basically achieved by adding more data to the training set, driving up both precision and sensitivity at the same time. In dictionary-based classification, it means getting more knowledgeable about the wording in the source. The task is easier when highly specific wording is characteristic for a type of ad, but in most cases, it is not possible to eliminate both types of errors.

For historians trained to treat sources in the most careful and exact way, such an inherent sloppiness of algorithmic approaches may seem hard to stomach. But it would be naïve to assume that evaluating and sorting hundreds of thousands of ads by hand could lead to a flawless and consistent result, either: Imperfections are inevitable when a very large collection of texts is interpreted and structured; it simply necessitates procedures to estimate and control errors, and to be aware of them in interpreting the data. – And there are, apparently, huge advantages of our approach of classifying big historical data by “dynamic tagging”:

1. First, while individual human decisions to tag remain implicit, undocumented, liable to individual understanding of text and hence inconsistent, dictionary-based tagging is fully transparent, reproducible, and consistent. This is an advantage also over supervised machine learning approaches, which build on a manually classified training set of considerable size.

2. Second, like supervised machine learning, dynamic tagging allows for tremendous economies of scale: once the classifier is developed, the only difference between tagging 100 and 100,000 ads is computing time.

3. Third, other than typical SML classification methods, dynamic tagging works fine for specific classes with lower case numbers. In fact, the more specific a class, the easier it is in the dictionary-based approach to achieve high precision and sensitivity.

4. Fourth, this allows for a bottom-up approach in some contexts, like deriving a classifier for furniture by building a set of smaller classifiers for separate types of furniture (table, cabinet, bed, etc.), and then uniting those more specific tags under the umbrella term “furniture”. Consequently, against 37 labels in our manual classification, we have currently 186 labels and counting. Employing such a fine-grained classification in manual tagging would make the process disproportionally slow and error prone.

5. Fifth, a manual classification is an ex-ante classification, in effect irreversible, and as such must be universal to serve all kinds of research questions. Dictionary-based classification scripts can be developed dynamically as the analysis unfolds. They can be modified, extended, and rerun anytime, without the need to build a new sample of (thousands of) manually classified ads as necessary in an SML approach. This enables an iterative process of refining hypotheses and refining heuristics, and it leaves the door open for any kind of future research questions.

For the purpose of this paper, to identify patterns of household behaviour in advertisement activity, use is made of all postings in the Avisblatt, and a range of subject-related tag filters that have been developed within the different parts of the project so far. ^[11] As the dynamic tagging will evolve still a bit further in the project, results here may not be ultimate.

2 Household Budgets and Income Elasticity

For a large part of the households in Basel, reading the Avisblatt became a routine part of everyday life. Advertising something in the paper, however, was usually not. It mostly signalled a smaller or larger break with everyday routine: the need to find new housing (or new tenants), to find a new job or employ someone new, the event of losing or finding something of value or importance, the decision to remove a piece of furniture or clothing from the household and sell it, or the wish or need to add such a piece.

There are exceptions to the rule: advertisements to promote a service or a shop could be placed routinely. Up to 1760 (when the authorities stifled the practice), lottery collectors produced a steady stream of ads to keep readers involved in the recurrent draws of the class lotteries. And as Anna Reimann shows in her research, there is a significant shift in the ads of the later decades towards an exchange of goods in shops, as opposed to classified ads for one-off transactions. Still, the placement of promotional ads was far more irregular, less recurrent, and less dense than we are used to from how today’s advertisement industry operates. In many cases, promotional ads were prompted by a change of address about which customers needed to be notified, and the many one-off promotional ads are less a testimony of everyday routine, but a pointer towards a new business or one that needed to address slumping sales.

So overall, most ads in the Avisblatt are indicative of a change or even break in everyday life for the advertiser. The individual reasons for it were manyfold, oftentimes a matter of individual fate (as in the case of losing objects, jobs, housing), yet in many cases a matter of the household budget becoming suddenly tighter, or wider. Especially the expenses for basic consumables were far less steady than today, with the very high share of the income most households had to spend on basic foodstuffs, and the prices for those fluctuating strongly. This implied the need for most households not only to constantly redistribute expenses, but given the necessity of expenses for basic needs, also adapt its sources of income.

In the late 18^th and in the 19^th century, most households in Central Europe had to spend half or more of their budget on food. There are essentially two ways to establish this empirically, and to determine consumption patterns of households more generally: either aggregating case studies on the microlevel of individual household budgets, or determining (with the help of trade statistics, data on revenue from indirect taxes and duties, etc.) the overall consumption of a (national or urban) economy, weighing it with current prices, and turning it into per capita or per household data. ^[12] An early Swiss example to calculate the average expenses per household is the attempt by Johann Heinrich Waser, who in 1778 estimated the annual consumption of Zurich, and divided it among the 1,972 households of the city. According to his estimate, 51.6 percent of an average household budget went into food. ^[13] In a similar way, Ludwig August Burckhardt calculated some consumption rates (but not a complete budget) for the city of Basel in 1826. ^[14] The main problem with this kind of calculation is that the average consumption pattern in fact is an average of all the quite different consumption patterns across society, and that is not representative of any one household in the society. It is not the consumption pattern of an average household in the middle of the middle-class: the calculated average consumption pattern includes expenses on jewellery and wooden shoes, because rich households bought jewellery, and poor households bought wooden shoes, but middle-class households likely bought neither.

The share of the budget spent on more essential needs, especially (basic) foodstuffs, tends to be lower, the higher the income of that household. This observation, known as Engel’s law, was made by the statistician Ernst Engel in an 1857 paper, in which he tried to estimate the value of the annual consumption and production in Saxony. ^[15] He used the extensive empirical data provided by Édouard Ducpétiaux for Belgian working-class budgets and the broader but rather eclectic collection of budgets prepared by Frédéric Le Play to derive general statements regarding income, budget structures, and their relation. ^[16] He then used those to make his estimates for Saxony. A core finding was that the most impoverished part of the Belgian working-class households, who could not subsist without public support, had to spend 70.9 percent of their expenditure on food, the households independent of outside support but unable to build savings 67.4 percent, and those who could subsist securely and save for retirement 62.4 percent. ^[17] Engel then inferred that Saxon working class households of the latter type would spend about 62 percent of the budget on food, a middle-class household 55 and a wealthy household 50 percent. ^[18] In 1848, a survey amongst local agricultural societies in Prussia had tried to establish what a typical local farm working household would need to subsist; in the resulting average budget, 61.2 percent of the expenditure is on food (or feeding animals providing for the household’s consumption). ^[19] For further comparison: Hendrik Fischer has calculated from a very extensive set of budgets that later on in Imperial Germany, households with an annual expenditure under 900 Mark spent on average 62.8 percent on food, those with 900 to 3.000 Mark (the vast majority of households in his sample) 54.6 percent, and those above 3.000 Mark less than 40 percent. ^[20]

Since the ground-breaking work by Ducpétiaux, Le Play, and Engel in the 1850s, establishing and analysing household budgets became of high interest to economists and statisticians interested in social conditions and policies, especially the conditions of the poor (the Social Question). Many studies were essentially case studies of specific budgets as reflected in single housekeeping books, but it was also attempted to collect data more broadly, through surveys or – mainly since the 1890s – by giving out housekeeping books to a larger number of households, especially working-class families. ^[21]

For the specific case of Basel, two representatives of the Historical School of Economics played a role in fostering research into local household budgets. Karl Bücher, a former participant in Ernst Engel’s statistical seminar at the Prussian Statistical Office in Berlin, held the chair for political economy and statistics at Basel University from 1883 to 1890. Mostly focused on historical demographics, he became a member of the expert committee for the 1888 Swiss census, co-organized the census in the canton Basel-Stadt and was commissioned with the housing enquete of 1889. ^[22] In this context and under his initial guidance, a local clerk called Carl Landolt, nephew of Wilhelm Liebknecht and sympathetic with the condition of the working class, started a survey of working-class budgets which he finished and published independently of Bücher. ^[23] They later clashed over methodological questions, and over who initiated the project and could claim merit for the conceptual work. ^[24]

The bigger impact came from Stephan Bauer, an Austrian-born, renown expert on social issues who held the chair for political economy at Basel University from 1899 to his death in 1934. ^[25] He was General Secretary of the International Association for Labour Legislation (the main predecessor of the International Labour Organization) and Director of the Associations’ office, from its establishment in Basel in 1901 to its dissolution following the founding of the ILO in 1919. Bauer was keenly interested in household budgets; he contributed the article on this topic to the first three editions of the famous Handwörterbuch der Staatswissenschaften. ^[26] In Basel, with the help of a union leader, he acquired housekeeping books from four families over several years, and entrusted them to one of his PhD students, Fritz Krömmelbein, as the material for his 1910 dissertation. Further housekeeping books were added and analysed by Bauer’s group in a 1917 anthology, which was part of a general survey by the Verein für Socialpolitik on price surges and real wages. It had ensued from a joint paper of Bauer and Irving Fisher in 1912, calling for an international survey on the cost of living. ^[27] Continuing along those lines, Emil Notz published an excellent dissertation supervised by Bauer in 1925, in which he studied long-term changes in purchasing power in Basel itself, harking back to the beginning of the 19^th century. ^[28]

With the help of economist and social statistician Fritz Mangold, co-founder and director of the Schweizerisches Wirtschaftsarchiv (Swiss Economic Archive) in Basel, Notz managed to find the budgets of two households in Basel for the year 1806. Household A is described as a family of four, a journeyman printer, his wife, and two underaged children; household B as consisting of three adults (a man, a woman, and their maid). Notz refers to A as a working-class household and B as a bourgeois household; in terms of income, status, and the share and proportion of the different kinds of expenditures, they mark the lower and upper end of the middle classes, and in the following are addressed as such. ^[29]

To estimate indicators for changes in discretionary income, these two budgets will be very valuable, and they can be complemented by a third one for a lower-class or poor household. In 1786, the Gesellschaft des Guten und Gemeinnützigen (Society for the Good and the Common Benefit), a charitable organization of Basel citizens to promote the education and wellbeing of the lower classes, undertook an inquiry into the living conditions of lower-class households in Basel. The inquiry commission did not collect actual budget data of exemplary households, they investigated the typical needs of different types of households and enumerated what they considered the expenses for a livelihood adequate to social status. The aim was to estimate the weekly income necessary for poor people to stay out of existential crisis. ^[30] While such data is far from being as empirically sound and precise as that collected by the likes of Ducpétiaux and Landolt, it is sufficient for the purpose of this analysis. Among the different household types discussed, type C, the case of a married couple without children in a flat of their own, is the most detailed one (and frankly, the least poor). Consisting of a couple in a flat of their own, it is also the one most comparable to the two 1806 households, only that in the 1806 cases, one of the couples has two children and the other no children but a maid. To make the generic type C household from 1786 more comparable to that, the consumption of cloth and food (other than meat) are increased 50 percent, to emulate the additional consumption of two children. ^[31]

Fig. 1

Expenditure Structure of Three Basel Households in 1786/1806

In this snapshot, the upper middle-class household spent 55.2 percent, the lower middle-class household 61.9 percent, and the poor household 61.2 percent of its budget on food – quite in line with the other estimates discussed above. The expenses for bread, beef, and butter alone required about a third of the income in each case; but the split between bread on the one hand and beef and butter on the other is 80:20 for the poor household, 59:41 for the journeyman and his family and 17:83 for the wealthier household (its three adults had, on average, each a pound of meat per day ^[32]). The share of different items in the two budgets is in line with what research has shown for comparable households, but it is important to note that those shares were not fixed: When the prices for single items changed, more (or less) money was needed for the food consumed, and this necessitated decisions of the household with respect to the rest of the budget. The ensuing redistribution was usually not made in equal proportion for each type of expense but depended on the income elasticity of the specific need.

The income elasticity of demand denotes how much the consumption of a specific good changes in relation to a change of household income. If income decreases 10 percent, but the consumption of bread decreases only 3 percent, then bread has an income elasticity of 3 percent / 10 percent = 0.3. Goods with a positive elasticity smaller than 1 are known as “necessary goods”: their consumption does not decrease or increase much when income changes, because they are deemed necessary regardless of income – bread is a case in point. Goods with an elasticity larger than 1 are “superior goods” or “luxury goods”, because they are cut disproportionally from the shopping list when the budget tightens and are put on it when income becomes more than sufficient to cover basic needs.

When household income changes, it usually results in a complex of substitution processes. Those substitutions can take place within a category of expenses, but there will also be shifts between major categories, like an increasing share of “food” expenditures corresponding with a decreasing share of “clothing”. To empirically determine income elasticities in a specific historical context, a lot of data on prices, income, and household budgets are necessary, so it is almost impossible to come up with robust numbers for any part of Central Europe, maybe all of Europe, before 1850. The closest approximation to our case of Basel might be the impressive study of Fischer on consumption in Imperial Germany, built on a database of 5,000 household budgets from more than 150 sources. ^[33] He pinpoints the elasticity of food at 0.6, clothing 0.8, housing 1.1, and other expenses 2.0. ^[34] In other words, food and clothing were necessary goods, housing a superior good by a hair’s breadth, and everything else a superior or luxury good.

Those numbers are calculated from differences in budget structure between households of different income classes at the same time. If we consider the behaviour of a household as a reaction to changes in income or, which will be the main point here, changes in prices of different goods, then time becomes an important part of the equation. It is useful to distinguish between adaptations of budgets in the short run versus those in the long run. The latter can be interpreted as households moving from one class of income, or one type of consumption pattern, to another. Shifts between different classes of expenditure are then described by the inter-income-class elasticities calculated by Fischer.

But how do household react immediately to an external shock like a rise in bread prices, likely in the hope that the bread price increase is only temporary? The consumption of food, and especially basic foodstuffs, cannot change that much, it is quite inelastic. But the cost of housing can also not be changed easily within a few weeks, as that requires moving into another location. Thus, in the short run, housing has a very low income elasticity, in the long run, it does not. On the other hand, one can usually hold off on clothing for some time, so in the short-term, income elasticity of clothing will have been notably larger than 1. Looking at changes in time, the order of necessities as suggested by the inter-household calculation of Fischer can become reshuffled, dependent on the time horizon under consideration.

3 Food Prices and Discretionary Income

To link changes in advertisement activity with changes in discretionary income, two types of data series need to be constructed: (a) the number of advertisements of a specific type printed in the Avisblatt, which is easily obtainable in our R package by simply counting ads with the pertinent classification tag for each timeframe; (b) one or more indicators for discretionary income of households in Basel. The compilation of the latter kind of series requires to gather suitable price data and, with a view on typical household budget structures, to determine a function to convert them accordingly.

An initial decision must be made on the timeframe used: shall weekly, monthly, quarterly, or yearly data be compiled? The counting of adverts is equally possible, and requires equal effort, for any of those timeframes. For historical price data that will be transformed into a discretionary income indicator, series of yearly prices are usually easier to come by. But the properties of the resulting data structures are a still more important consideration: If we count how many ads of a certain kind were posted in each month or even each week (= issue), there would be more gaps in the series than for quarterly and monthly data. It could cause problems with classes of less common ads, of which maybe only one or two per month are observable on average, so that the randomness of observations has a notable impact on the accuracy of the analysis. Counting all ads published in a year solves this problem, but then we only have 116 data points in total. Breaking down the 116 years into four periods (to discover changing patterns over time) would leave us with only 29 data points per period, which makes it harder to find significant correlations. More importantly, it would also be impossible to discern timing effects (for example, if advertisement activity in a certain segment changed in tandem with discretionary income, but with a few months delay). Having data series on a quarterly basis thus appears to be a good compromise, if sufficient price data of that kind can be found.

A wide range of price data from Basel has been compiled and published, covering different parts of the 116 years in question. For the early 19^th century, data has been provided by Emil Notz in the 1920s, ^[35] for the 18^th century by Markus Mattmüller and his group in the early 1980s. ^[36] Unfortunately, all this published price data is on a yearly basis and there remain some gaps over the 116 years in question. Fortunately, the current prices of grain and some other foodstuffs, mostly butter and meat, were regularly published in the Avisblatt itself; the editor of the paper obviously took it from official market reports (mainly the weekly Fruchtpreiszettel) and price tariff announcements. This data was retrieved as part of the SNSF Avisblatt project. The weekly grain price data has only short gaps, mostly in the 1740s and 1790s, some of which could be filled by revisiting the Fruchtpreiszettel. There are also bread prices for stretches of time, which with a view on household budgets, is a much more pertinent information than grain prices. Using the decrees regarding bread tariffs, ^[37] it was possible to calculate a bread price from the grain prices; checking them against the occasional published bread prices proved the calculations correct. ^[38]

Butter prices were continuously published in the Avisblatt until 1757 and then again from 1805, the gap could for the most part be filled by data from the Fruchtpreiszettel. The availability of meat prices, on the other hand, is a problem.

There is dense data from 1730 to 1754, and quarterly meat tariffs were published with few exceptions from 1818 from 1837 – apart from that, only occasional meat prices could be drawn from the Avisblatt. In the future, the data might be complemented from other sources. For now, it is notable that butter prices and beef prices developed very similar (R²= 0.951). As about 95 percent of the variability of the beef price data is shared with the butter price data, the linear regression is for now used as a proxy to fill the gaps – even though those prices were more in step in the longer run, and not so much from quarter to quarter. ^[39]

How can those prices be used in combination with the three household budgets discussed above, to construct indicators for discretionary income? First, a simplification is necessary: household income is taken to be constant. For households with wage income, this is not unreasonable, most short-term changes in their budget will not have arisen from changes in earnings, as wages were comparatively stable. For example, the daily wages of carpenters, bricklayers, and mason journeymen employed by the city were adjusted by the municipal payroll office (Städtisches Lohnamt) only five times during the 116 years under consideration here (1733, 1767, 1793, 1807, and 1842). ^[40] There is no indication of wage increases that were felt by a large part of the Basel households at the same time, which otherwise could be taken into account here. Still, the assumption of constant income is not very realistic: For many households, especially the more well-off, income stemmed not from wages; craftsmen and tradesmen households lived by the (fluctuating) revenue of their business.

Yet this simply means something which is fairly obvious anyway: Many of the changes in individual discretionary income which prompted a household decision, which in turn lead to a posting in the Avisblatt, were not caused by shifts in the prices for basic foodstuffs, but changes in income, other parts of the budget or, mostly, reasons outside the budget. Changes in the number of advertisements of certain types cannot be fully explained by food price changes; but this of course is in no way the intention of this study. Rather, changes in food prices can be considered as a cause for a change in discretionary incomes that is much more generally felt. It parallels individual decisions and strategies, and creates visible patterns of household and advertising behaviour, which would otherwise be indiscernible in the white noise of contingent individual decision making and advertising.

Finally, what about changes in income that were not independent of food prices, and thus not random from the viewpoint of the analysis? This is not a problem, either. On the one hand, attempts to counter changes in discretionary income by generating additional earnings is exactly the kind of economic decision that hopefully becomes visible in the advertisement data. On the other hand, there is also a systematic relation between general economic climate and the revenue of craftsmen and tradesmen households. But this relation should for the most part be of a Keynesian nature, rather reinforcing the tendencies induced by changing food prices, in a downward spiral: When the budget for most households tightened due to increasing food prices, the effective demand of those households for goods and services suffered, in turn reducing the revenue for most of those who served that local demand. Their discretionary income decreased even further, likely with some delay. Inversely, there should have been an upward spiral when food prices declined, and discretionary incomes expanded.

Discretionary income, to be sure, is a somewhat loosely defined variable: the remainder of the disposable income (i.e., income after taxes) after accounting for essential expenditure. But what expenditures can be deemed essential? There is no objective answer, it is a matter of definition or convention. For the purposes of this study, the components of household budgets are distinguished into three groups. First, the expenses for basic foodstuffs – here taken to be bread, beef, and butter – are considered as independent variable, i.e., at what drives changes in discretionary income. They are here, by definition, viewed as essential expenditure. The potential substitution between bread and meat consumption (when one became relatively more expensive than the other) is ignored here, as that would overcomplicate the model without much gain in the results. Second, the amount of money needed for rent, heating, and lighting is rather income inelastic at least in the short run (housing can usually not be changed instantly), so just as the expenditure for other food, it is considered essential expenditure. In lieu of data, both shares are taken as constant. Third, everything else: that what is left of the budget, when the cost for food and housing are accounted for, is defined as discretionary income, our dependent variable: the amount of money spent for clothing and all other expenses. These are the budget items with the highest (short-term) income elasticity, those most prone to change when the budget tightened or expanded.

Taking all this into consideration, three series of discretionary income data can be constructed as follows: The share of discretionary income in the total budget at the time of origin is already given in the budget structure itself (in figure 2 marked as a dot). It is then recalculated how much the respective amounts of bread, beef, and butter consumed in 1786/1806 by the different households did cost at different points between 1729 and 1844. The discretionary income share is then taken as what would have been left of the total budget at those points, after accounting (a) for those variable cost for basic foodstuffs, and (b) the cost for other food and housing (assumed here also as a fixed sum, i.e., always the same as in the year from which the budget originated).

Fig. 2

Share of Discretionary Income in the Total Budget of Three Generic Basel Households

It is apparent that the general quarterly changes of discretionary income induced by food price changes were notable, and thus relevant for households’ budget decisions; at some points very markedly, so: the turmoil of the Napoleonic period shows, as well as a food price spike during the constitutional crisis that led to the separation of the old canton Basel in 1832/33, and of course the impact of the hunger years 1770/71, 1771/72, and especially 1816/17. Life for the lower classes was precarious, and food price surges were an existential threat, especially if there were no savings: the discretionary income of the generic lower-class or poor household (di.P) became negative several times, the di.LMC in the spring of 1817 as well. Finally, di.UMC, albeit mostly not much larger than di.LMC, remained always positive and was less volatile: the upper middle-class household’s lesser dependence on bread signals not only a higher level of living standard, but also of living standard security.

The downward trend in all three discretionary income series may appear to signal that overall, living standards regressed slightly – but that downward trend is purely an artefact of the construction of the three series. In the calculation, the total size of each of the three budgets is assumed to be invariant over time. This is clearly not realistic, as income – when it came in the form of wages – may have been constant over long stretches of time, but certainly not over a span of 116 years. The price level of basic foodstuff increased overall, and as a reaction to these inflationary tendencies, wages were adapted sooner or later. As the inflation of food prices is reflected in the discretionary income series, but any rise in income is not, it evokes the unfounded impression that discretionary income generally decreased over the years. But depending on how incomes developed, any trend is conceivable here.

Not knowing the real trend would be most unfortunate, if we were mainly interested in the size of the share that the discretionary income had of the total budget. Yet the aim here is to construct a data series that reflects the impulse of basic food prices on discretionary income, so it is only the change of the share from quarter to quarter that is relevant. For the analysis, we use the first differences of each discretionary income share series: not the shares themselves, but the difference between shares from one quarter to the next. In the same way, we also use the first differences of the advertisement counts: the change in the number of adverts of a certain kind from one quarter to the next. Both kinds of series are stationary, ^[41] they do – among other properties – not contain a trend, which is a crucial pre-requisite for performing a correlation analysis. ^[42]

4 Detecting Patterns of Stabilizing Discretionary Income

It is not possible to establish the social class of the advertiser for each advertisement directly, since a very large part is anonymous, and even those that are not usually contain insufficient information for such an exercise. For that reason, for any class of advertisements under consideration, we separately examine a potential relation to any of the three series of discretionary income shares (di.P, di.LMC, and di.UMC), which reflect the changing budget situation for the three generic household types. If a relation shows for one or two, but not all three, it points to a class-specific advertisement behaviour.

Taking a series of quarterly changes in advertisement activity and one of the three series of quarterly changes in discretionary income share, we test for a significant correlation between the two and examine if the advertisement activity intensified when the discretionary income share increased (as indicated by a positive correlation), if the activity decreased (negative correlation), or if no such connection shows (i.e., correlation not significantly different from 0). Initial analysis shows that quite often, changes in discretionary income do have an impact on advertisement activity, but not necessarily at the same time; the effects may be lagging one or more quarters. Consequently, we employ cross-correlations and multiple regression.

For a specific class of advertisements, let N _i be the normalized ^[43] number of such advertisements in quarter i; then the quarterly change in advertisement activity is y_i := N_i – N_i-1. For any of the three generic types of households, let D _i be the share of discretionary income in quarter i (as depicted in figure 2), and x_i := D_i – D_i-1 the quarterly change. Then we estimate quarterly change in advertisement activity to be

In the following tables, for each subset of advertisements analysed, N/q denotes the average number of ads per quarter (mean of N _i for all i), to show the absolute level of advertisement activity in that case. For each subset, a multiple regression is attempted for each of the three types of households. For each such regression, if any of the estimated coefficients proves significantly different from zero, the following is given:

1. R²: adjusted R-squared of the multiple regression;

2. sig: The significance level of the multiple regression, as indicated by the F-test (· ≤ 10 percent, * ≤ 5 percent, ** ≤ 1 percent, *** ≤ 0.1 percent chance to obtain an R²of at least the magnitude observed, if it would be true that there is no influence of discretionary income changes on advertisement activity after all);

3. lag: weighted average lag time, to indicate if advertisement activity reacts fast or slow to changes in discretionary income; it is calculated by estimating four single regressions, one for the cumulated change from each lag (0, 1, 2, 3) separately, and then weighing lag-time by the respective R-squared (0×R₀²+ 1×R₁²+ 2×R₂²+ 3×R₃²) / (R₀²+ R₁²+ R₂²+ R₃²);

4. dir: the predominant direction of the relation, as an indicator if the impact of discretionary income change on advertisement activity is mostly positive (less income implies less ads, more income more ads) or inverse (less income implies more ads, and vice versa); it is determined by weighing the sign (-1, 0, or +1) of the coefficients β₁, β₂, β₃, β₄ by R₀², R₁², R₂², R₃²as calculated for L, and denoting if the result is positive or negative.

For a start, let us consider all the postings in the Avisblatt, broken down into offers and requests. A first, very general hypothesis would be that rising food prices and the consequent decrease of discretionary income for most inhabitants of the city reduced economic activities beyond the acquisition of food, and with that impending shrinkage of the economy, the hustle and bustle in the Avisblatt dyed down somewhat, as well. In that case, the correlation between discretionary income and advertising activity should generally be positive (less income, less ads), both for requests as a direct reflection of lessened demand, and offers as reflecting greater pessimism, adapting to lower effective demand, and generally cutting expenses like spending on advertisement.

But on this most aggregate level, there is a positive relationship only for requests and income changes as experienced by middle-class households. The pattern is most clear and significant for the generic upper middle-class household, the relation becomes weaker for the lower middle-class, still less significant for the poor, and here it is also an inverse instead of a positive relation. The lag time of changes in advertising seems to be connected to the level of income, too. Given that changes in the discretionary income of generic poor household were driven mostly by bread expenses, those of the generic upper middle-class mostly by meat and fat expenses, and the lower middle-class household equally by both, all of this suggests a faster and more pronounced reaction to changes in beef and butter prices, and more of an initial wait-and-see for bread price changes.

Incidentally, as far as different results for different (income) classes are concerned, there are always two general factors at play: lower income households will have felt food price changes relatively more, so Avisblatt-related strategies to stabilize discretionary income should have been more pronounced; at the same time, the fee for an advertisement – itself an additional expense! – was felt more by lower-income households as well. ^[44] The net effect is unclear.

Contrary to the initial hypothesis, the frequency of offers in the Avisblatt has in general an inverse relation to income changes: offers were made more frequently in worse and less frequently in better times. This is also slightly more pronounced for the more bread-dependent households, but with a similar lag-time across all household types. As N indicates, offers accounted for about 75 percent of the ads, so this pattern is the dominant feature here. That the hustle and bustle in the Avisblatt intensified in times of tightening budgets suggests that households used it as a tool to mitigate crises.

Breaking down ads into more specific classes, different patterns of crises mitigation appear. When the budget tightens but expenses cannot be cut accordingly, households run into losses – which is only bearable as a temporal problem and if there are savings or saleable items for rainy days. Consuming those savings does leave no mark in the Avisblatt, but occasionally it might have become necessary or advisable to touch on a nest egg and liquidate an asset. One type of asset, for the middle and upper classes, were securities, especially public bonds, and shares in interest funds. And indeed, while requests for securities through the Avisblatt seem not sensitive to the economic climate, sale offers of single securities increased with the tightening of the purse strings – although the overall number of such ads is comparatively small (see table 2). More commonly, distress sales will have involved material possessions such as a piece of clothing or furniture; often in advancing a sale that would have been made at a later date anyway.

Another way to create extra income would be heightened industriousness. The vending of consumables is a good example; there is a vast number of offers of self-made cheese, butter, herbal tea, prepared fish, spiced wine, donkey’s milk, etc., and of goods procured from afar such as coffee, tea, sugar, and tobacco. This class of industrious ads exhibits a very clear inverse relationship to discretionary income, especially that of the middle-class households.

In part, the offers of clothing are also reflective of industriousness; what is striking though is the positive relation between the discretionary income of upper middle-class households and the requests for attire (see table 2): How can the effective demand for clothes increase when discretionary incomes shrink? The most likely explanation is a process of substitution, a switch from having new clothes tailored to buying used: the Avisblatt is an especially suitable platform to provide a market for second hand durables. The substitution strategy seems to have been confined to clothing, though, as the results for ads concerning furniture and non-textile household items show: the requesting ads are even more numerous than those for clothing, but not significantly correlated to income changes. This might be linked to higher durability and income elasticity of furniture and most household items. The wear and tear of clothes is high, and the urgency to replace them despite a tightened budget higher than for a cabinet or tableware.

Running out of savings and being unable or unwilling to create additional income, the borrowing of money could have been considered an option. There indeed is a significant relation between loan requests and the economic climate, but it is positive: lending intensifies when discretionary incomes increase and dwindles when spendable money gets scarcer (see table 3). The crux of the matter is the type of loans that are advertised: the Avisblatt loan market is generally not concerned with microcredits, or sums that one could deem characteristic of consumer credits, but loans of at least several hundred, often several thousand guilders: with a view on the direction of the correlation, they must be understood as investment loans for business endeavours.

Not all the patterns that are observable can be easily explained. Unsurprisingly, job offers decline when times get tougher and are more numerous in good times, but employment requests were also more common in good than in bad economic climate, which seems counterintuitive. The Avisblatt job market does for a large part concern servants and apprentices, and apprenticeships were more of an investment than an opportunity to earn additional income, for usually the apprentice’s parents had to pay a fee to the master for the education as well as room and board (Lehrgeld). But this accounts only for a part of the effect; the patterns of the servant job market point to the need for an in-depth study of that segment beyond the kind of analysis made here. The same is true for the patterns in the housing and property markets: in both cases, both sides of the market tend into the same direction, but housing for rent became more important with overall tighter budgets, while the sale of houses and flats was more common during good times. Detecting patterns of household behaviour not only verifies and falsifies ex-ante hypotheses about specific household strategies; it also helps and forces to develop new hypotheses.

A final perspective to touch on here is the persistence and change of patterns over time. Breaking down the 116 years into four periods of 29 years, it appears that overall, relations between discretionary income changes and advertisement activities show mostly in the second and third period (see table 4).

One possible explanation could be that classes of ads that have presented themselves as especially sensible to the economic climate dominated in those periods, while initially and in the end, less sensitive ads were more important; but the shares of different classes of ads in the Avisblatt do not change in such a dramatic way. This leaves two other explanations, changes in audience composition (or audience sensitivity to food prices) and changes in household strategies. It is indeed likely that initially the paper was predominantly used by the most well-off in town, whose discretionary income was not very sensitive to changes in basic foodstuff prices, before the user group expanded more into the middle and lower classes. But it can be ruled out that the audience shifted back to the well-off in the nineteenth century; and while slight gains in real income for the middle classes are conceivable (this needs further examination), it would be highly surprising if that rendered strategies of stabilizing discretionary income against food price changes redundant. All in all, the findings point to changes in household strategies.

The overall change of patterns for consumables, clothes, and furnishings through the four periods is similar to what can be seen in table 4, since those are large classes of ads that contribute much to the overall picture. For the large class of employment ads, the relation noted in table 3 is significant for the second to fourth period, most strongly in the third. The housing market stands out as a particularly wild case (see table 5): from period to period, the relation between income changes and the inclination to advertise changes not only direction, but also the side of the market – another pointer that a detailed case study of the housing market is in order.

It is notable that by differentiating the data into four periods, the significant relations exhibit a much higher R², i.e., a bigger part of the quarterly variation in the number of ads becomes explainable by the quarterly changes in discretionary household incomes. As mentioned before, it is and cannot be the aim to try and account for all or even only most of that variation, as it is mostly white noise from contingent decisions of individual households to post an advert. But food price changes are so important for households that occasionally they cause similar decisions by many. Changes in discretionary income for the generic upper middle-class household account for 1.7 percent of the quarterly variation in offers over the 116 years (see table 1), but the relation is in fact significant only for the second of the four period, 1758-1786, and during that time accounts for a staggering 32 percent.

This is an extreme example, but the problem is the same throughout: for hardly any class of ads and type of generic household, discretionary income changes explain more than five percent of the variation in advertisement activity over 116 years, but a higher and often much higher share during shorter periods, against hardly anything in between. In short, the household strategies observable here were not persistently operative at a high level, but quite impact-ful during the time that they were. In case studies of markets or types of exchange that feature in the Avisblatt, such as the housing market, it will therefore be advisable to detect more precisely when exactly the strategies observed were effectively operational. This can be done by calculating R²for a moving window, for example the twelve years or 48 quarters from the second quarter in 1729 ^[45] to the first quarter in 1741, from Q3 of 1729 to Q2 of 1741, and so on, up until the twelve years from Q1 of 1833 to Q4 of 1844. As an illustration, here is the moving window for the totality of all offers and all requests:

Fig. 3

Impact of Income Change on Advertisement Activity (R²), 12 Year Moving Window

In this picture, all the different patterns from different household strategies, from consumables and clothes to employment and housing, are baked together. While this obscures the more specific patterns, especially the more atypical ones, it does present an overall impression of how the Avisblatt became used again and again, by many households at once, to stabilize their discretionary income.

5 Conclusion

Data-driven research into the economic and social history of households has a longstanding history. And for a long time, such research focused on a description of social structures, and conditions of the different parts of a community or society. With research concepts like the consumer revolution or industrious revolution, interest shifted to the changing behaviour of households. Studying probate inventories in large numbers became one way to investigate this beyond single case studies, through the material result of the actions of households.

This article suggests that in utilizing classified ads in great numbers, intentions and decisions of many households can also be observed more directly, in the form of aggregated patterns of advertisement behaviour. This is the case under circumstances that put a great number of households before a similar problem at the same time, so that decision-making otherwise contingent and independent from others becomes more aligned, leaving visible traces. The most important common problem households face is the change in the prices of basic foodstuffs, with marked consequences for the discretionary income left after spending on basic needs. Did advertisers react to tighter budgets by posting less or more ads? By distinguishing offers from requests, and between different classes of ads, evidence for a number of different household strategies to stabilize discretionary income could be found in the data: from distress sales and increased industriousness to shifting from new to used goods. Some of the patterns observed require and suggest broader studies to interpret them in a meaningful way, such as in the case of the housing market.

The household strategies were not stable over time. They were not persistently operative and continuously visible in the advertisement data, but again and again in a very notable way. Some of the patterns change over time, and all the strategies appear transitory: some were not yet evident in the first few decades of the Avisblatt, and some not evident anymore in the last few. And as far as they were, they were enabled only with the creation of the medium. Both the inception and the end of the Avisblatt signal that as a specialized platform for everyday exchange – next to markets, shops, auctions etc. – it itself was a transitional arrangement in the shift from early towards industrial modernity.

The insights gathered from the general approach here come at a notable price. To digitize a serial source in the discussed form is time- and resource-consuming. While the script-based approach allows for a sustainable, flexible, and open use of the data for research, it requires skills that are usually not too common among qualitative historians and which need to be acquired, i.e. handling data using scripts, understanding code in programming languages like R or Python.

The reward for these investments is enticing however: Having a serial source represented in structured digital form, as big historical data, allows research to zoom in and out, to utilize both the bird’s eye and the worm’s-eye view, to detect general patterns and to bring details to life and into contexts in a “thick description”, to bridge micro and macro histories. With different independent researchers contributing (asynchronously) to the classification, structuring, and analytical permeation of the source, research can be collaborative and synergetic in effect, without being bound by a rigid, coordinative organizational form. Big historical data allows the combination of the data-drivenness, methodological transparency, and analytical rigour of economic history as historical economics with the hermeneutical openness and depth of economic history as cultural history. Moving the discipline forward in the digital age might just require, and lead to a greater such convergence.