Law and Economics of Language Model Development: Empirical Examination of Corporate Strategies and Vaporware Claims

1 Introduction

Large-scale language models (LLMs), including ChatGPT, have the potential to instigate significant transformations within the economy and society, presenting numerous considerations from the vantage point of law and economics (Note 1). In light of these applications, the paper significantly contributes to the writing process, particularly in structuring and aiding the articulation of ideas (Note 2). However, this paper delves into characterizing corporate strategies surrounding ChatGPT and LLMs through the lens of ‘vaporware’, an existing concept in law enforcement. It is crucial to note, as this paper intends to elucidate, that while certain behaviors indicative of vaporware were observed, they do not conclusively fit the classification. Subsequently, we analyze the stock market as a monopolistically competitive arena, conducting an event study on stock prices for firms announcing the initiation of LLM development, including ChatGPT. This analysis aims to ascertain whether there is an increase in cumulative abnormal return (CAR). The findings on CAR are then scrutinized to determine their trajectory. The overarching aim of this study is to unravel the vaporware characteristics in the development of LLMs, including ChatGPT, and to assess the necessity for regulatory measures.

LLM are a type of computer program used in the field of natural language processing (NLP). It is designed to perform tasks related to the understanding and production of human language. LLM are trained using large amounts of textual data as training data. Training data includes text collected from a variety of sources, including documents on the Web, blogs, news articles, books, and Wikipedia. The evolution of this LLM has improved performance on a variety of NLP tasks, including machine translation, automatic summarization, sentence generation, and question answering. LLM are being used in many application areas, including information retrieval, customer support, and automated sentence generation.

In deep learning, the phenomenon of generalization error decreasing as the number of parameters is increased is known as the inverse of overlearning (overfitting). Overlearning is a state in which the model overfits the training data and the generalization performance for new data decreases. While increasing the number of parameters improves the expressive power of the model, and models with more parameters can learn more complex functions, there is also an increased risk of overlearning, which is high performance on training data but poor performance on unknown data. However, with a sufficient amount of additional training data, the model is now able to learn more complex patterns by increasing the number of parameters. With this particularly large data set, models with more parameters capture more patterns and features, potentially resulting in better generalization performance, and indeed, ChatGPT based on this has been very successful.

However, it has been pointed out that these LLM are in principle developed and operated in English, and that their accuracy in other languages is not as high as in English (https://ja.stateofaiguides.com/20200817-nlp-beyond-english/). This has resulted in strategic actions by companies trying to address this gap. Specifically, a series of announcements by large Japanese companies regarding the establishment and operation of LLM in Japanese have arisen (It is organized later in Table 1).

Even among high-tech firms, including R&D-oriented firms, such announcements that they have begun development are of a somewhat unusual kind. Usually, announcements of R&D results are made to stimulate demand or lead to further investment. This is partly because such announcements of research results at an early stage, when they have not yet been commercialized, are sometimes called vaporware and have been viewed with suspicion.

Vaporware refers to computer hardware or software that was announced to the public in the computer industry but was delayed or never actually manufactured or officially cancelled. Historically, the U.S. Department of Justice was notably accused of doing this by IBM in 1969 in United States v. IBM. There, IBM announced the release of the System/360 Model 91 after an IBM competitor, Control Data Corporation (CDC), released a computer. This announcement resulted in a significant drop in sales of CDC products. However, because the System/360 Model 91 was not released until two years later, the U.S. Department of Justice accused IBM of doing this intentionally. (Note 3, Note 4) Similar lawsuits have been filed against Kodak, AT&T, and Xerox. (Note 5) Also, U.S. District Court Judge Spokin vociferously opposed the practice during the 1994 settlement hearing in United States v. Microsoft Corp. “Vaporware is, on its face, a deceptive practice, and everyone in the business community knows it.” said Spokin. (Note 6)

In the United States, it is illegal under Section 2 of the Sherman Antitrust Act of 1890 to announce a product that does not exist in order to gain a competitive advantage, but in practice, few hardware or software developers have ever been convicted for this type of conduct. False or misleading announcements intended to influence stock prices are also illegal under the Securities Fraud Act in the U.S. and the Financial Securities Exchange Law in Japan. Several companies have been charged in court with knowingly making false announcements to gain an advantage in the market.

It is not easy to directly observe and evaluate whether a Japanese company’s announcement of its efforts to build LLM in Japanese constitutes vaporware or a misstatement in its securities report, since there are a wide variety of factors to consider, including the perceptions of the people involved and objective technical standards. Therefore, we would like to examine the vaporware nature (misstatement character) of this announcement by understanding consumer perceptions of such an announcement, as well as the impact of this LLM initiative on the economy as a whole.

In order for vaporware to be established, it is assumed that a certain degree of market dominance exists (Note 7). However, with regard to LLM technologies such as ChatGPT, it is extremely difficult to even consider the possibility of a strong business, especially since it is a software and web service in a high-tech market, and its usage is still being explored around the world, and it is also extremely difficult to investigate how consumers perceive it. It is also extremely difficult to investigate how consumers perceive the software.

In response to this situation, we consider stock market trading of the shares of each company, in which each company is differentiated from the others and, while recognizing the trends of other companies, the company is the sole provider and consumer interest in it is demonstrated. The stock of each company is considered to be a market in which market dominance exists (Note 8), a place for trading in monopolistic competition, where the company is the sole provider but competition is taking place and no excess profits are being generated. In this market, we attempt to verify whether each company’s LLM activities are indicative of vaporware-like trends.

In this case, we consider each firm’s press release as a shock that reveals information that was not previously available in the market, and examine how the shock is reflected in the stock market by examining the change in CAR through a stock price event analysis.

The efficient market hypothesis is a hypothesis that all available information is fully reflected in market prices. In other words, in the stock market, since all available information is reflected in prices, asset prices follow a random walk, and since it is not possible to predict future price changes using currently available information, investors cannot earn more than their expected returns. Stock price event analysis is the idea that there is a possibility that an event, say the announcement of certain information, could have an impact on price formation.

The conclusion of this study was that LLM development announcements were not perceived as vaporware at all by the companies, and that their announcements were received calmly by the market. This is a robust result, as it is the same even when the event window is changed or when the control group is taken into account. Therefore, it is highly likely that LLM development will be constrained by market mechanisms, and it is difficult to see a clear rationale for the immediate introduction of competition and industrial policy regulations.

This paper is organized as follows. Section 2 reviews prior studies, particularly the recent consideration of the economic impact of LLMs, particularly ChatGPT. It then summarizes the Vaperware study, which was a major discussion in law and economics at the time of the Microsoft litigation. Section 3 summarizes the methodology and data in this paper. It also discusses descriptive statistics and summarizes the methodology for the stock price event analysis. Section 4 summarizes the results. In addition, the robustness of the data is examined based on several methods. Section 5 is the conclusion.

2 Previous Study

Regarding the impact of LLMs around ChatGPT in recent years, the following are some of the most prominent discussions of artificial intelligence (AI) and skills, starting with Autor as a particularly prominent discussion here from an economics perspective. The study offers a unified analysis of the growth of low-skill service occupations between 1980 and 2005 and the concurrent polarization of US employment and wages. The study hypothesizes that polarization stems from the interaction between consumer preferences, which favor variety over specialization, and the falling cost of automating routine, codifiable job tasks. Applying a spatial equilibrium model, the study corroborates four implications of this hypothesis. Local labor markets that specialized in routine tasks differentially adopted information technology, reallocated low-skill labor into service occupations (employment polarization), experienced earnings growth at the tails of the distribution (wage polarization), and received inflows of skilled labor (Autor and Dorn 2013). In light of this, therein,the study discusses how recent AI advancements, largely fueled by machine learning’s predictive abilities, affect jobs. It highlights the importance of understanding the distinction between prediction and decision-making roles. The paper illustrates how AI’s influence on labor can differ when automation is used for decision-making versus when it aids human decision-making (Agrawal, Gans, and Goldfarb 2019). This has been shown, among other things, recently by the study that explores the potential impact of LLMs like GPTs on the U.S. labor market, emphasizing the amplified capabilities of LLM-powered software. Using a unique method, the study evaluates jobs based on their alignment with LLM capabilities. The results show that roughly 80 % of U.S. workers could see at least 10 % of their tasks affected by LLMs, with around 19 % possibly having 50 % of their tasks impacted. These effects could touch all wage brackets, with higher-income jobs at higher risk. Importally, these effects are not limited to high-growth industries. With LLMs, about 15 % of all tasks could be done significantly quicker without quality loss. This rises to between 47 % and 56 % when considering LLM-powered software, implying substantial economic effects. The study concludes that LLMs, like GPTs, show characteristics of general-purpose technologies with wide-ranging potential implications (Eloundou et al. 2023).

In addition to this, the direct impact of LLMs such as ChatGPT is discussed: The study presents a method that utilizes online job ad data to evaluate the likelihood of emerging technologies becoming General Purpose Technologies (GPTs) – rare innovations with substantial economic impact. This technique helps identify which technologies might have a broad economic influence versus those that will be beneficial but remain narrow enabling technologies. This method, with at least five years predictive power, is distinct from traditional patent-based approaches. Applied to 21 emerging technologies, it indicates that machine learning and related data science technologies are quite likely to become GPTs (Goldfarb, Taska, and Teodoridis 2023). The other study observes that early uses of artificial intelligence technologies (AITs) were narrow yet highly profitable, and their widespread application could lead to an economic boom. Firms replicating these early applications could gain strategically, a pattern seen in previous information and communication technology rounds. For AITs to become a general-purpose technology across sectors, new innovations are necessary. The paper explores potential innovation paths for this outcome, those that have stalled, those currently in progress, and future possibilities. Interestingly, early AIT usage, in both commercial and technical sectors, succeeded without needing Digital Transformation. However, all potential innovation paths identified necessitate Digital Transformation as crucial steps (Bresnahan 2023). In addition, the following more specific verifications have been made. The study examines the impact of an AI-based conversational assistant on 5179 customer support agents. The tool enhances productivity by an average of 14 %, particularly benefiting novice and low-skilled workers, with little impact on experienced or high-skilled workers. The AI model appears to share the tacit knowledge of more skilled workers, aiding less experienced workers. The study also shows that the AI tool improves customer sentiment, lessens the need for managerial intervention, and boosts employee retention (Brynjolfsson et al. 2023).

This present study does not examine these aggregate effects on labor or the direct effects of technology from the events here; the study examines how LLM is used where it is one of the business strategies, and the actual effects when it may cause harm, and the law to regulate it based on these effects in terms of law and economics to advance the discussion.

Furthermore, this research extends its scope to include traditional marketing strategies, highlighting the exigency of law enforcement and regulatory measures in this domain. A critical aspect of this study is the exploration of the legal and economic implications of vaporware, a practice characterized by premature product announcements that lack substantiation at the time of disclosure. This practice’s potential harm and the need for regulatory oversight are scrutinized, drawing attention to the 1995 Microsoft antitrust case. In this landmark case, Judge Stanley Sporkin’s initial rejection of a consent decree brought to the fore the contentious nature of vaporware. While the judge’s ruling was subsequently overturned, it catalyzed robust discussion surrounding the potential antitrust ramifications of vaporware strategies (Note 9).

The notion of fraudulent vaporware marketing, which is classified as predatory under the Sherman Act, threatens to impede fair competition. Nevertheless, vaporware announcements that are innocently made with strategic intent ought not to be construed as antitrust violations, given their prospective efficiency benefits (Prentice and Langmore 1994). Furthermore, vaporware, defined specifically within the context of software that is announced but not subsequently delivered as expected, has ignited contentious debate over its potential anticompetitive effects. However, given the deleterious impact on a firm’s reputation, the probability of firms deliberately issuing false product pre-announcements appears minimal. As such, vaporware may not pose a significant concern for antitrust authorities (Levy 1997). From an economic standpoint, vaporware – a strategy whereby a firm inaccurately announces a new product, often to deter potential competitors – warrants analysis. The argument proposed is that vaporware operates as an equilibrium strategy within a signaling game and can exert adverse effects on consumers. Despite this, the overall welfare implications of vaporware remain equivocal, and the need for further research is thus underscored (Haan 2003). Similarly, the widespread practice of “vaporware,” where firms pre-announce products prior to their market availability, is a notable trend within the computer industry. Another study develops a model suggesting that such announcements, despite constituting “cheap talk,” may yet convey valuable consumer information. That study interrogates the welfare implications of this practice, the potential network effects, and its prospective influence on firms’ innovation strategies, thereby advocating for further research concerning the nexus between product pre-announcements and R&D incentives (Choi, Kristiansen, and Nahm 2010).

This study is intended to examine the vaporware nature of the company’s LLM efforts and its response. We advance our understanding of the possibility and reality of how new technologies are used in old marketing strategies.

3 Methodology and Data

4 Results

4.1 Estimation Formula and Results

In this section, the estimation formula is as follows (equation (1)).

(1) SP i , t = α i + β i × NIKKEI t + ε i , t

where i be a subscript for each company (i = CyberAgent, Fujitsu, Hitachi, and Nippon Telegraph and Telephone), SP be the stock price of each company, and NIKKEI be the Nikkei average, and β_i for each company is calculated from data for the two months before the announcement date. The subscript t indicates time and is taken daily. The CAR is then obtained by subtracting the actual stock price from the assumed stock price using β_i from the day following the announcement date. This is done until one week after the announcement date, and the values are compared. α and β are the coefficients to be obtained. e is the error term. Here, instead of using stock price returns and market returns, the CAR equivalent is estimated by calculating the estimated stock price and estimating and accumulating the stock price returns from it.

The α and β for each company are as follows Table 2:

Table 2:

The α and β for each company.

		α (S.E.)	β (S.E.)	Adjusted R2
1	CyberAgent, Inc.	262.685 (157.263)	0.031 (0.006)	0.427
2	Fujitsu, Ltd.	10619.070 (1386.118)	0.256 (0.048)	0.403
3	Hitachi, Ltd.	−5336.157 (639.894)	0.451 (0.023)	0.911
4	Nippon Telegraph and Telephone Corporation	2803.168 (316.748)	0.043 (0.011)	0.273

Table 3:

Relationship between stock price movements of each company and control group before and after the announcement date and the announcement date dummy variable

Dependent variable:	Press release Dummy
Method: least square	n = 21	n = 21
	Coefficient (S.E.)	Coefficient (S.E.)
C	−10.737	−17.828
	(5.7750)	(12.7109)
CyberAgent, Inc.	0.001	0.018
	(0.0113)	(0.0152)
Fujitsu, Ltd.	0.127a	0.069
	(0.0316)	(0.0557)
NTT	−0.063	−0.039
	(0.0456)	(0.0614)
Gree, Inc.		−0.029
		(0.0420)
NEC Corporation		0.063
		(0.0412)
Toshiba Corporation		0.123
		(0.1763)
KDDI		−0.014
		(0.0659)
R-squared	0.907	0.929
Adjusted R-squared	0.884	0.882
Akaike info criterion	−0.453	−0.342

1. ^aIndicates 1 % significance.

Here, we consider the announcement date as the event window and the CAR up to two weeks later. There have been some indications regarding stock price volatility due to information leaks prior to an event (e.g., Talay, Akdeniz, and Kirca 2016). For example, CyberAgent made similar announcements in January and March 2023, and the possibility that such information is reflected in stock prices cannot be denied. However, in this study, we only consider the elimination of uncertainty in the announcement of official views as one event, and therefore, in principle, we consider the announcement date of each company’s official announcement as the event date. Based on this, we then examine the event date and the cumulative abnormal return after the event date for each company. However, one limitation of this study is that the possibility of stock price fluctuations due to such information leaks cannot be completely ruled out.

Based on this, the CAR for each company were calculated as the difference between the estimated and realized values, and their two-week trends are shown in the following chart (Figure 1).

Figure 1:

Changes in CAR.

The above chart shows the cumulative ratio (%) of the CAR to the stock price by day, starting from the day after the LLM development announcement date, which is 0. According to this chart, although Fujitsu’s ratio is slightly positive on the first day, all other ratios are negative, indicating that consumers and investors do not positively evaluate each company’s announcement of LLM development efforts.

For CyberAgent’s CAR, a t-test based on results up to day 4 yields a t-value of approximately −3.152 and a p-value of approximately 0.9901, and a t-test based on results up to 2 weeks later yields a t-value of approximately −5.127 and a p-value of approximately 0.99997. Both of these results indicate that there is insufficient evidence to support the null hypothesis that the CAR is positive. In other words, the conclusion is that it is not statistically significantly greater.

Similarly, for NTT’s CAR, a t-test based on the results up to day 4 yields a t-value of approximately −4.092 and a p-value of approximately 0.9968. A t-test based on the results up to 2 weeks later showed a t-value of approximately −7.462 and a p-value of approximately 1.0000. For the Hitachi CAR, a t-test based on the results up to day 4 yielded a t-value of approximately −5.770 and a p-value of approximately 0.9994. A t-test based on the results up to 2 weeks later yielded a t-value of approximately −10.972 and a p-value of approximately 1.0000. Furthermore, a t-test based on the results up to day 4 for Fujitsu’s CAR yielded a t-value of approximately −1.938 and a p-value of approximately 0.9497. A t-test based on the results up to two weeks later yielded a t-value of approximately −0.997 and a p-value of approximately 0.8346. All of these company results indicate that there is insufficient evidence to support the null hypothesis that the CAR is positive. In other words, the conclusion is that it is not statistically significantly greater.

The following chart (Figure 2) illustrates the stock price trends of each company after the date of their LLM development announcements.

Figure 2:

Trends in stock prices.

The graph shows that no company’s stock price has increased significantly since the announcement of the LLM development.

Therefore, the announcement of LLM development was not perceived as vaporware by the companies, and the announcement was received calmly by the market. It is highly likely that the actual launch of the service and its market acceptance is recognized as something to come. If the announcement by high-tech companies that they will develop products incorporating innovative technologies has any effect on rivals, for example, it is likely to be perceived by the market as an increase in the number of competitors of the same type competing in technological innovation and that there will be even fiercer competition in the future.

4.2 Robustness Check

The robustness of this estimation is examined from the following two points. The robustness of this estimation is examined from the following two points: changes in the event window and the use of control groups.

As for the change in the event window, the next figure plots the CAR from the fourth day after three days have passed (Figure 3), instead of looking at the cumulative abnormal earnings from the day after the LLM development announcement date.

Figure 3:

CAR from the 4th day.

The same trend can be seen in this figure. In other words, CAR are negative for all firms, suggesting that the market is not receiving the firms’ announcements positively.

Next, as a control group, we would like to look at the stock price trends of Japanese competitors during the same period compared to the stock price trends of each company. The companies that are comparable to each of the firms are as follows

1. GREE, Inc.: Like CyberAgent, Inc., GREE is active in the fields of Internet advertising and social games. GREE develops and operates mobile social games and operates a digital advertising business.

2. NEC Corporation (NEC): Like Fujitsu Limited, NEC is a world-renowned company in the fields of information and communications technology, computer products, and telecommunications infrastructure; NEC provides computers, network equipment, security solutions, and cloud services.

3. Toshiba Corporation: Like Hitachi, Ltd., Toshiba Corporation is a company with a wide range of businesses in the fields of electrical equipment, information and telecommunications equipment, and energy. Toshiba offers products and services such as home appliances, information systems, and infrastructure systems.

4. KDDI Corporation: Like Nippon Telegraph and Telephone Corporation, KDDI Corporation is a company that operates in the fields of telecommunications services, Internet access, and mobile communications. KDDI provides cell phone services, fixed-line telecommunications services, data center services, and cloud computing services.

The following graph compares these companies with CyberAgent, Fujitsu, Hitachi, and Nippon Telegraph and Telephone, discussed here, by picking up 10 days before and after the date of LLM development announcement, along with the change in the Nikkei Stock Average. The graph shows the Nikkei Stock Average for each of the 10 days before and 10 days after the LLM development announcement (Figure 4).

Figure 4:

CyberAgent, GREE, and Nikkei Stock Averate.

As can be seen from these graphs, there is no clear difference between the companies considered here after the announcement of LLM development and the control companies, with the exception of Fujitsu. Incidentally, when we looked at the relationship between the indexed stock price and the stock price in the 10 days before and after the announcement using a dummy variable that takes 0 before the announcement date and 1 after the announcement date, no company (except Fujitsu when no control is used) was significant at 5 %.

(2) P R t = α + β i × Company i , t ( + γ j × Company j , t ) + ε i , t

In equation (2), the subscript t represents time, i represents each firm, j represents each dummy firm, PR is the announcement date dummy variable, and Company represents the stock price of each company. α, β_i, and β_j are the coefficients to be obtained, and ε is the error term (Figures 5 –7).

Figure 5:

Fujitsu, NEC, and Nikkei Stock Average.

Figure 6:

Hitachi, Toshiba, and Nikkei Stock Average.

Figure 7:

NTT, KDDI, and Nikkei Stock Average.

From the robustness of the results, it can be assumed that the announcement of each company was received calmly by the market, and no vaporware-like effects were found.

There are various possible reasons why the announcement of the LLM development did not lead to an increase in the company’s valuation. In the following, for example, the limited impact of a company’s announcement about developing an LLM can be condensed into three main reasons:

1. Communication and Relevance Issues: Ineffective communication of the project’s benefits and relevance, coupled with a failure to align the development with the company’s core business and strategic goals.

2. Market and Competitive Factors: The announcement’s lack of distinction in a competitive market and the technical complexity of the LLM, making it hard for non-experts to grasp its importance.

3. Company Reputation and Timing: Skepticism arising from the company’s past performance and credibility, exacerbated by inopportune timing of the announcement amidst more pressing market issues or negative news.

However, it is reasonable to assume that one or more of such reasons could not have been at work and that market efficiency was demonstrated and legitimate valuation did not result in an increase in valuation.

In addition, in one episode, in March 2023, Baidu announced the Ernie bot, the Chinese version of ChatGPT, to the market’s disappointment and a 10 % drop in its stock price. In fact, even financially strong companies like Baidu lag behind some U.S. companies in the generative AI space. It is conceivable that the market’s lack of appreciation for the development of generative AI by Japanese companies may be due to their perceived lack of strength to challenge the monopoly of certain companies. If this antitrust issue is to be discussed in the area of generative AI, it may be better to address the issue of monopoly by a few companies rather than unfounded concerns about vaporware (NOTE 10).

5 Conclusions

This study empirically analyzes the presentation of corporate strategies for LLM development, such as ChatGPT, from a law and economics perspective. LLM derived its performance by increasing the size of the data set to a gigantic size. Therefore, in principle, it is possible to produce innovations that show similar performance in languages other than English. For this reason, several Japanese companies have announced the development of ChatGPT and similar models in Japanese. In this study, we examine whether these firms’ announcements can be positioned as vaporware, which has once been an issue in the antitrust debate. Specifically, we conducted a stock-event analysis using the date of the announcement as the event with regard to the fact that the announcement regarding the development of LLM, including the Japanese version of ChatGPT, did not have the vaporware nature that is likely to be a problem for antitrust in the high-tech sector. That is, a survey was conducted to verify that the firm’s announcement regarding LLM development was not vaporware and that the firm’s CAR was not elevated. The results showed that there was no increase in CAR with respect to these announcements, no vaporware element was observed, and the market reaction was confirmed to be quite calm in a robust manner. This may indicate that market mechanisms are calm regarding LLM development investments, making it difficult to justify current restrictions in competition policy and promotional measures in industrial policy (Note 12).

The implication of this study is that this study raises important questions about the validity of existing industrial and competition policies that might unduly restrict or impede such technological advancements. It suggests the need for a more nuanced understanding of the market dynamics surrounding the development and adoption of advanced artificial intelligence technologies. Future studies may build upon these findings to further investigate the role of market dynamics in shaping AI development strategies, and how regulatory frameworks might adapt to support beneficial competition and innovation.

The limitation of this study is that it primarily focuses on the reaction of financial markets to the announcement of LLM development, CAR as an indicator of whether the announcement was perceived as vaporware. Therefore, it does not account for the actual follow-through and execution of these projects, which is crucial for understanding whether these initiatives yield the promised results and benefits. Another limitation is the lack of analysis of the impact on the stock market of the leakage of information about the start of development. Furthermore, it primarily concentrates on Japanese companies, which may limit the generalizability of the findings to other cultural or economic contexts. It is also limited in its ability to address potential long-term effects of these technologies on market competition, consumer welfare, or broader societal implications. Future research may aim to address these gaps by investigating the post-announcement performance of companies, broadening the scope to include a more diverse set of global firms, and examining the specific details and impacts of different competition policies.