A Framework for Understanding Automation Propensities in the Job Market Carson Song ’19 Brown University Applied Math - Computer Science 1 2 1 Introduction With the phrases “artificial intelligence” and “Deep Learning” surfacing in the news along- side projections of rampant automation, this paper aims to explore Deep Learning in order to clarify its ability to disrupt the job market. We propose that automation occurs at the task level and that the probability a job is automated can be understood in terms of how likely it is that each task required within that job becomes automated. The paper uses a three-dimensional task space into which any task can be mapped and analyzes how Deep Learning performs in each segment of this task space. Then, a job can be separated into its component tasks, these tasks can be analyzed within our framework, and the automa- tion propensity of the job can be estimated in terms of the summation of its task-level automation propensities. What is artificial intelligence? In this study, we use the word automation to describe the process by which an artificially intelligent tool replaces human labor. As technology has advanced and its presence in the average home increased, the threshold quantifying artificial intelligence (AI) has consistently been raised higher and higher such that it always marks the boundary between human and current technological capabilities. To the populace of fifty years ago, the concept of a machine that immediately delivers the answer to any entered question would certainly have been considered artificially intelligent. Today, by the demystifying process of familiarity, we understand this machine, a search engine like Google, to be the product of complex algorithms–not a truly sentient system. Because of the subjectivity and relativity surrounding the phrase artificial intelligence, we will simply define AI as any technological tool that can solve a task. This assumption will set aside subjectivity and semantic complications and allow us to explore more objectively the current state of automation potentials. A Task-Based Analysis Importantly, we defined AI as a tool that can complete a task – not necessarily perform a job. Any particular job is a composition of distinct tasks. Often, this composition is diverse and requires a breadth of expertise. Consider a fast-food cashier. This employee performs a number of tasks throughout the day including conversing with a customer, taking payment, and delivering food. Instead of exploring the automation propensity of a fast-food cashier, we discuss how effectively AI might automate each in- dividual task. We choose to make this distinction because automation will fundamentally occur at the task-level [7]. AI may be able to effectively automate only some of the required tasks in a particular job, and even if an entire job can be automated, the replacement will not be one-to-one with a machine taking the place of a human laborer; instead, a suite of machines will work together to perform the composition of required tasks [7]. 3 Figure 1: The Three-Dimensional Task Space The Three-Dimensional Task Space This study utilizes a three-dimensional frame- work within which any task may be mapped. AI performance throughout the task space will be explored which will deliver an understanding of how likely it is that any partic- ular task becomes automated by existing technology. The three dimensions of the task space are Task Complexity, Environmental Rigidity, and Level of Social Interaction (Figure 1). We define the opposing ends of Environmental Rigidity as mechanical and dynamic. Mechanical tasks operate according to a static set of rules and there is a singular task ob- jective that does not change. Dynamic tasks, in contrast, require adaptation to changing rules, uncertainty in environmental interactions, and variability in task objective. In the fast-food cashier example, calculating the amount due for a sale is a relatively simple task bounded by a strict set of rules and requiring little social interaction (Simple, Mechanical, Non-Interactive). Handling an angry customer, in contrast, is a complex social task that requires one to balance customer happiness with company costs in an environment subject to changing circumstances (Complex, Dynamic, Interactive). Organizational Overview In section 2, we establish the status quo of AI capabilities and outline a baseline assumption that mechanical, non-interactive tasks can be automated 4 up to a degree of complexity. In section 3, we show that Deep Learning has nullified this complexity threshold. Complexity of tasks is no longer the limiting factor; instead, only the quantity and quality of available data limits AI automation [7]. With this in mind, the complexity axis is removed from the picture, and we conclude all mechanical, non- interactive tasks are subject to high risk of automation. In section 4, we show that Deep Learning is ‘narrow’ AI and struggles to adapt in dynamic environments. In section 5, we show that despite limitations in social contexts, interactive tasks can be automated on a limited and practical scale, especially under the assisted model. This will complete our analysis of automation propensities at the task level as catalyzed by Deep Learning technology and will provide a framework for understanding the according effects on the job market. 5 2 The Status Quo: AI Can Automate Simple, Mechanical, and Non-Interactive Tasks Familiar Automation We will first investigate the automation of familiar mechanical and non-interactive tasks. Consider a dirty stack of plates and utensils and the task of cleansing them. Before the dishwasher was invented, this task was always completed by a human. Now, one must simply load dirty plates into appropriate slots, so the dishwasher can run its automated cleaning process. Though the specific load composition may change, the machine runs the same hot water cycle during each use. The start and end points of the task are always consistent, the machine’s purpose never changes, and the dishwasher is never required to interact socially with those in the kitchen. Though trivial in its familiarity, the dishwasher is an example of the successful automation of a simple, mechanical, and non-interactive task. Looking beyond this household example, we consider robotics in industrial production. In car assembly, for example, the production line is split into distinct mechanical tasks with standard inputs and outputs. Robots replace human labor because the machines can precisely repeat these standardized tasks tirelessly. Because the task of shaping a piece of sheet metal is simple, mechanical, and non-interactive, it is unsurprising that the task has been successfully automated. Automation in this category is associated with a lack of surprise because AI solutions in this segment have been integrated into everyday life in the form of household appliances and industrial tools. Because of this integration, society takes for granted the status quo–that existing AI technology can automate simple, repetitive, non-interactive tasks. Though automation in this segment of the task space is often unsurprising, there are limits; a single robot cannot construct an entire car. Instead, the production line must be broken down into tasks small and simple enough to be automated by current robotics technology. Cashier software can certainly handle the simple arithmetic required to calculate charges incurred by retail shoppers. It would be much more difficult, however, to design a software that uses consumer trends to regulate pricing in order to balance stock levels while maximizing margins. Though both tasks are non-interactive, bounded by a rigid set of rules, and defined by a singular objective, automation of the second task seems unlikely due to its greater complexity. According to the status quo of AI capabilities, automation of mechanical and non-interactive tasks appears to be limited by the third axis, task complexity. Higher complexity seems to deter automation and necessitate human oversight. 6 Figure 2: 28x28 rendering of a handwritten ‘7’ (left). Pixel intensity matrix (right). Pixels with typically high intensities in images of ‘7’s are circled (right). [2] 3 Deep Learning Nullifies the Limiting Effects of Task Com- plexity Introduction to Deep Learning: Classifying Handwritten Digits This section will show that developments in Deep Learning technology and the according advancement in AI capabilities has eliminated task complexity as a barrier to automation. Thus, tasks that are mechanical and non-interactive in nature are poised for automation, regardless of their complexity. To demonstrate the power of Deep Learning, we consider the task of handwritten digit recognition. The bounds of the problem are undoubtedly rigid: there are only 10 digits to consider and each has a standard from which a handwritten version deviates. We assume the task is digitized and standardized such that the machine is fed 28x28 pixel images that are cropped and centered (Mechanical). Social interaction is not a part of this task (Non-Interactive). We will show, however, by attempting to construct an effective classification algorithm, that the task is much more complex than it might at first appear (Complex). Mimicking analysis performed by Charniak, we consider the case of classifying a ‘7’ using rules-based matrix computing–what is called a ‘heuristic’ approach [2]. Observing Figure 2, we could begin by assigning some system of rules that checks for high pixel intensity in pixels (8, 8), (10,19), and (22, 14) [2]. Additional rules checking for other high intensity and low intensity regions typical of a ‘7’ could successfully categorize this particular image. This approach fails to endure the variability innate to human handwriting. If a ‘7’ is slightly slanted or shifted within the frame or the length of the horizontal part of the ‘7’ 7 is slightly shorter than normal, these exacting rules will fail to properly classify the digit. One might then try to adapt the rules to parse relational probabilities based on important regions in the pixel matrix. This approach would be exceedingly difficult for one digit yet alone for all ten, and we see that the heuristic approach for this problem becomes unwieldy very quickly [2]. In contrast to this heuristics programming approach where a system of rules must be contstructed, Deep Learning is a universal function approximation architecture requiring no expert knowledge [2]. We are searching for a function that takes 784 parameter inputs (the 28x28 pixels) and outputs one of the ten digits. Deep learning is well-suited to this problem. A Deep Learning program is iteratively trained on paired input images and correct classifications, and the program’s internal workings are progressively shifted to give correct classifications with higher probabilities. In the results that follow, we used training data pulled from the MNIST database [6]. Using the basic architecture provided by Charniak, we tested a model with just one feed-forward layer trained a single time over 60,000 training images and achieved an accu- racy of 85% on a 10,000 image test set (MNIST) [2]. This architecture can be improved by expanding it to two layers separated by an activation layer. In under a minute, this network was trained and delivered an accuracy of 95% on the MNIST database. When the architectures incorporates convolutional layers that enable the architecture to recognize regional patterns in the image, the network delivers an accuracy of 98% on the MNIST database. With a basic architecture that incorporated zero expert knowledge of the problem, Deep Learning manages to complete a very complex mechanical, non-interactive task with 98% accuracy. Financial Time Series Prediction We will next investigate how Deep Learning can be applied to a prediction task on financial time series. The problem is defined by rigid bounds and a singular, constant objective, and requires no social interaction. Despite a favorable categorization as a mechanical and non-interactive task, the high degree of noise in financial data makes the task notoriously complex and historically difficult. Despite this complexity, Fischer and Krauss demonstrate that Deep Learning achieves predictive efficacy surpassing that of typical financial models derived using expert knowledge on stock market behaviors. [5] The study employs a Long Short-Term Memory (LSTM) model–a Deep Learning archi- tecture designed for sequence learning tasks such as speech recognition or building predictive text models [2]. After applying an LSTM network on a large, liquid, and survivor bias free stock universe, including all S&P 500 constituents from 1992 to 2015, the network’s per- formance was benchmarked by comparisons with three other predictive algorithms: a state of the art random forest algorithm (RAF), a standard deep neural network (DNN), and a standard logistic regression (LOG) [5]. 8 Figure 3: Construction of Input Sequences for the LSTM architecture [5] The LSTM’s task boils down to determining a relationship between predictive signals (the features) and future returns (the targets). The network’s sequence of inputs consists of a single feature: the standardized one-day return at consecutive points in time. Figure 3 demonstrates how sequenced data can be constructed from the historical standardized one-day returns on stock 1, s1. Sequence 1 is passed through the LSTM which returns a predicted sequential output, Sequence 1*. The difference between predictive Sequence 1* and the actual successor, Sequence 2, serves as the loss along which the model is corrected. In Fischer and Krauss, the four algorithms were tasked with forecasting the probability that each stock s out-/underperforms the cross-sectional median over a period of time [5]. The ranking of these probabilities delivered an assessment of how well the stocks were valued and was used to develop a trading strategy of going long on the top k stocks while shorting the bottom k stocks. Analyzing the results on a small portfolio (k=10) demonstrates the LSTM is a suitable and effective model that outperforms even the state of the art random forest algorithm (RAF) and the general market (MKT) [5]. We analyze the performance along three characteristics: i–daily return, ii–daily risk, iii–annualized risk/return (See Figure 4 for the complete dataset): i) The LSTM architecture delivered the best mean return of 0.46% return per day. ii) LSTM had the second lowest VAR ( daily Value at Risk) behind only the RAF model. iii) LSTM achieved the highest Sharpe ratio at 2.34 (the Sharpe ratio scales excess return by standard deviation to deliver a metric for return per unit risk). [5] While the LSTM architecture passes the eye test, Fischer and Krauss conducted a Diebold-Mariano null test and verified the superiority of LSTM forecasts (p-value = 0.0143, 0.0037, and 0.0000) [5]. We conclude that the LSTM model (a Deep Learning architecture) demonstrates clear analytical competence in a mechanical and non-interactive problem involving complex, non- linear relationships. In fact, the LSTM’s mean return of 0.46% per day indicates a statisti- cally significant challenge to the semi-strong form of market efficiency and suggests a Deep Learning solution could be effectively deployed in this domain [5]. 9 Figure 4: k = 10 portfolio performance characteristics. [5] 10 Tumor Detection in Lymph Nodes To further bolster the claim that Deep Learning negates the limiting effects of task complexity, we will explore how Deep Learning performs in the field of medicine. Steiner et al demonstrates the use of LYNA, a Deep Learning algorithm that locates tumor cells in lymph node imaging for the purpose of identifying metastatic breast cancer [11]. Though the task of analyzing slides is repetitive and highly structured, accurate classification requires years of training and experience due to the com- plexity of human tissue samples. The three categories of images that were reviewed in the study included images dis- playing micrometastasis, macrometastasis, and negative characteristics. Instances of mi- crometastasis contain a tumor between 0.2mm and 2.0mm in diameter [11]. These tumors are more difficult to detect, and recent analysis demonstrates that with hematoxylin and eosin (H&E) dye alone, pathologists deliver a suboptimal sensitivity of only 38% detec- tion [11]. To counteract these limitations in practice, immunohistochemistry (IHC) can sometimes be used to produce better imaging and improve diagnostic sensitivity, but the IHC stain is costly, time consuming, and has varying use across clinics. Steiner et al’s LYNA is used to analyze these same H&E stained lymph node im- ages. LYNA analyzes slides and produces outlines of probable tumor regions. Patholo- gists were presented with a set of images either with or without the assistance of LYNA outlining and were asked to report whole-slide level findings–negative, micrometastasis, or macrometastasis–according to the largest tumor deposit in an image. Figure 5 juxtaposes the Unassisted(A) vs. LYNA-assisted (B) views of a lymph node image with IHC (C) stain used for tumor verification. For micrometastases, the tumor variant that is both smaller and more difficult to detect, LYNA-assisted sensitivity was 91.2% while unassisted sensitivity was 83.3% (statistically significant at 95% confidence) [11]. No statistical difference between assisted and unassisted sensitivity in the other two categories (negative and macrometastatic cases) was observed. These results are shown in Figure 6. The Deep Learning model increased tumor sensitivity for the category with the lowest detection sensitivity. Given the steep consequences sur- rounding false negatives in tumor detection, this sensitivity increase provided by LYNA is valuable and significant. Although this case study discusses LYNA in terms of its use as an assistive tool and not an independent classifier, we conclude LYNA can effectively parse complex human tissue samples and accurately identify tumors. In fact, all of the false negatives in the macrometas- tasis category corresponded to a single image where pathologists failed to identify a tumor that LYNA did highlight with moderate confidence (Figure 7). LYNA’s effective ability to detect tumors serves as another example of Deep Learning’s competence in completing a mechanical, non-interactive tasks of high complexity. Deep Learning Nullifies Complexity Limitations Through these case evaluations, we observe that Deep Learning has the ability to nullify the challenges imposed by task 11 Figure 5: Suspicious regions of interest are highlighted in cyan for high-confidence and green for moderate-confidence. Both outlined regions are confirmed as metastatic tumor by IHC stain [11] Figure 6: Sensitivity performance with and without LYNA on the three slide categories [11] 12 Figure 7: Macrometastasis false negative by all participants despite moderate-confidence outline by LYNA (left - tissue sample, middle - LYNA overlay applied, right - IHC stained sample [11] complexity. Deep learning solutions excel in complex, mechanical, and non-interactive tasks such as prediction tasks on the stock market and tumor detection in human lymph nodes. We conclude that the threshold limiting automation of mechanical and non-interactive tasks is not the level of complexity. With quality training data, mechanical, non-interactive tasks of any complexity can be automated [7]. With this in mind, we remove the complexity axis, collapse the task space to two dimensions, and indicate AI’s performance capability in the mechanical, non-interactive segment (Figure 8). In the following sections, we will explore this updated task space and focus on automa- tion propensity of tasks in dynamic environments and tasks requiring social interaction. 13 Figure 8: The updated two-dimensional task space. Deep Learning can automate mechan- ical, non-interactive tasks of any complexity. 14 4 Performance Suffers in Dynamic Environments Deep Learning is Narrow AI In this section, we will investigate how Deep Learning solutions endure along the environmental axis. Moving away from mechanical environ- ments, tasks exit the realm of standardization and begin to operate amongst uncertainty and according to dynamic rules or objectives. We conclude that for all of its breakthroughs in completing complex (but rigid) tasks, Deep Learning is decidedly ‘narrow’ AI [7]. Deep learning solutions operate exceptionally well at the specific task for which they have been trained. We will show that deviation from the constraints as defined in the training en- vironment or alteration of the objective renders a Deep Learning solution useless. We investigate this phenomenon and its consequences for automation propensities within the dynamic region of the task space. The ability of a Deep Learning model to detect cancerous constructs in lymph node imaging suggests a mimicry of radiologist capability. We explore, however, whether this mimicry derives from a human-like learning process that can endure task alterations and introduction of uncertainty. We will determine whether Deep Learning shares these charac- teristics that allow for robust performance in dynamic environments. As a demonstrative case study, we analyze a Deep Learning powered controller for the game of Pacman often used as a benchmark for AI study [9]. Pacman Using Deep Q-learning, a reinforcement learning architecture, a Pacman con- troller is able to win 95% of games played on a static map [12]. Considering that Pacman operates in a mechanical environment on a static map with deterministic ghost behavior, it is unsurprising that Deep Learning completes this task competently (Mechanical, Non- Interactive). In reinforcement learning architectures, the model repeatedly plays the game and hones its decision making process based on how well it performs. One might assume that through this iterative training, the model would, like a human, intuitively learn facets of the Pacman game. A novice human player learns the rules of the game and basic evasion techniques by playing the game repeatedly. Changing the map does not significantly alter the human’s ability to play the game because the basic rules and evasion techniques still hold true and can be applied to the new map. To test how Deep Q-Learning ‘learns’ to play the game, we similarly stress the AI controller. When the Deep Q-Learning solution is tested on new maps, the model chooses near random movements, fails to obtain rewards, and fails to win [12]. We conclude that Deep Learning solutions do not demonstrate the same extrapolative ability that distinguishes human learning. This distinction is important in that, unlike human actors, Deep Learning solutions tend to fall apart when the task environment is changed. CartPole To expand upon the results demonstrated in the Pacman analysis, we use the game of CartPole, a simple ‘classic-control’ game that is part of the OpenAI Gym 15 Figure 9: The CartPole game configured with default parameters [1] repository [1]. In the game, a controller observes a state consisting of four entries: cart position, cart velocity, pole angle, and velocity at pole tip. At each time step, the controller must choose to move to the left or the right at a discrete force-magnitude. The controller earns one point for each time step that it keeps the pole upright–between the theta-threshold bounds. The CartPole game configured with the default parameters can be seen in Figure 9. We used an adapted version of Charniak’s Advantage Actor Critic (A2C) architecture to train a controller for this game [2]. In the A2C architecture, two Deep Learning models are used. The critic learns a function that assigns a value to any possible input game state, and the actor learns a policy for how to move based on the input game state. The difference between the actual rewards received by playing the game and the values calculated by the critic at each time step–the advantage–is used to train the two models. CartPole is such an easy task that after only 1000 training iterations, the controller played the game 100 times and scored an average 484.75 of the 500 possible points. With the controller’s ability to effectively play CartPole established, we assessed this model’s competency in playing modified versions of the game. By adjusting certain param- eters, we explore the solution’s robustness in dynamic environments. The following table summarizes the results of modifying just one parameter at a time (averaged over 100 trials): 16 PARAMETER Default 1 2 3 4 5 6 7 8 gravity 9.8 9.8/2 9.8 ∗ 2 - - - - - - pole-length 0.5 - - 0.5/2 0.5 ∗ 2 - - - - force-magnitude 10 - - - - 10/4 10 ∗ 4 - - theta-threshold 3π/45 = α - - - - - - α/2 α∗2 Av. Reward 484.75 491.34 298.39 425.26 480.93 191.14 406.78 285.37 486.45 We see that performance significantly suffers for trials 2, 5, 7 where the model scores less than 60% of the possible points. Increasing the gravity, decreasing the force-magnitude, and decreasing the theta-threshold all resulted in lower scores. It is somewhat unsurprising that the model suffers lower scores because these parameter changes make the problem more difficult. Next, we analyze the effects of changing two parameters at a time as summarized in the table below (Trials 1*-24*): PAR. 1* 2* 3* 4* 5* 6* 7* 8* g 9.8/2 9.8/2 9.8 ∗ 2 9.8 ∗ 2 9.8/2 9.8/2 9.8 ∗ 2 9.8 ∗ 2 length 0.5/2 0.5 ∗ 2 0.5/2 0.5 ∗ 2 - - - - force - - - - 10/4 10 ∗ 4 10/4 10 ∗ 4 theta - - - - - - - - Av. Rew. 481.84 494.28 213.62 285.09 470.6 449.91 75.77 336.72 PAR. 9* 10* 11* 12* 13* 14* 15* 16* g 9.8/2 9.8/2 9.8 ∗ 2 9.8 ∗ 2 - - - - length - - - - 0.5/2 0.5/2 0.5 ∗ 2 0.5 ∗ 2 force - - - - 10/4 10 ∗ 4 10/4 10 ∗ 4 theta α/2 α∗2 α/2 α∗2 - - - - Av. Rew. 290.76 493.89 137.01 389.39 184.83 154.34 187.97 476.09 PAR. 17* 18* 19* 20* 21* 22* 23* 24* g - - - - - - - - length 0.5/2 0.5/2 0.5 ∗ 2 0.5 ∗ 2 - - - - force - - - - 10/4 10/4 10 ∗ 4 10 ∗ 4 theta α/2 α∗2 α/2 α∗2 α/2 α∗2 α/2 α∗2 Av. Rew. 148.31 458.21 234.96 487.37 139.62 207.77 76.8 469.16 Observing trial 8*, there is significantly decreased performance when increasing gravity and increasing the force-magnitude. Observing trial 9*, performance suffers when gravity is decreased and theta-threshold is decreased. Trial 22* shows low performance when force- magnitude is decreased while theta-threshold increased; trial 23* shows low performance with these parameter changes reversed. 17 In each of these cases, we test the model under opposing parameter changes: one pa- rameter change makes the problem easier while the other makes the task more difficult. In effect, the problem difficulty remains relatively the same but the task environment has changed. With average scores of 336.72, 290.76, 207.77, and 76.8, the Deep Learning con- troller demonstrates ineffective performance under such conditions. In fact, trial 23*’s average reward of just 77 points is only marginally better than a controller that moves randomly at each time stop (Av. Reward = 24.71). Furthermore, the model scored about 30% of the possible points when the pole-length was shortened and the force-magnitude was increased despite the fact that neither of these parameter changes alone were associated with significantly decreased performance (trial 14*). These examples demonstrate that a changed task environment–not an increase in task difficulty–causes the Deep Learning controller to earn a lesser score. Implications for Real World Tasks The Pacman and CartPole examples demonstrate a limitation to Deep Learning that has profound implications for automation propensities within the job market. Deep Learning architectures struggle to perform in environments unseen in training. The cited Pacman study proposes that the model might generalize over multiple levels by training on a subset of these levels [12]. In other words, the model’s shortcomings may be negated by the acquisition of training data that encompasses all possible scenarios. This potential solution is impractical in real world applications. Consider a consulting agency tasked with providing marketing advice. It would certainly be possible to create a Deep Learning solution capable of effectively marketing one specific product within a particular context. Trying to build a data set that represents all possible future products and contexts is impractical if not impossible. But without this synopsis on which to train the AI, the program will fall short in its performance across varied use cases. Returning to a previously discussed application, the tumor detection algorithm, LYNA, would deliver nonsensical responses (or no response) to anomalies in human tissue outside of those encountered in training. LYNA cannot observe an image and detect generally unhealthy tissue, it can only detect those structures it was specifically trained to detect. We conclude that Deep Learning fails to operate effectively in dynamic environments where the model is faced with scenarios unobserved in training. We accordingly update our task space graphic to reflect this conclusion (Figure 10). 18 Figure 10: Deep Learning Fails to Operate in Dynamic Environments 5 Partial Automation of Interactive Tasks: The Assisted Model The Turing Test We will lastly investigate the propensity for AI solutions to handle tasks which require degrees of social interaction. We initiate this discussion by investigating the current intersection of AI capabilities and the Turing Test. In order to determine if machines could think, Turing proposed an imitation test wherein a judge would hold a conversation with both a machine and human and determine which was which. Due to limitations in robotics technology and human avatar construction, AI developers have slightly altered the test and instead tasked themselves with developing chatbots that are indistinguishable from humans conversing with keyboards. A recent study by de Kleijn, Wijnen, and Poletiek investigates the factors contributing to the perceived humanness of sentences and the performance of cutting-edge chatbots on this modified Turing Test [3]. Even when limiting the problem to textual conversation, passing the Turing Test proves to be an enduringly complex problem. With a p-value of 0.003, the study found judges to reliably recognize humans as humans and chatbots as chatbots [3]. In other words, the chatbots demonstrate poor performance on the modern Turing Test due to consistently less humanlike responses. Figure 11 shows chat bots responses are perceived to be less humanlike than responses by human counterparts. 19 Figure 11: Perceived humanness of question-response combinations delivered by all chatbot and human agents [3] Even though the best chatbots fail to pass the Turing Test, their noticeably less human- like communication does not render them useless. People do not have to be totally fooled to still have practical use for chatbots. A recent Forbes article begins, “Chatbots are killing customer service. Obliterating it, maybe.” [4]. The article cites a recent Computer Gener- ated Solutions (CGS) study and quotes Michael Mills, a senior vice president at CGS. Mills describes, “Consumers will use a chatbot when they have a pretty straightforward ques- tion...half of the [survey] respondents ranked chat as their top channel for quick customer service.” [4] The study also concludes that live specialists are needed to handle more com- plex cases when the interaction becomes more complicated and less routine. [4] This result is empirically justified by the rise of voice assistants in the form of Amazon Alexa devices and smartphone softwares like Siri. Consumers will converse with AI-powered agents for productivity benefits even if these tools are imperfect and feel machinelike. Considering the willingness of consumers to engage with chatbots and other interactive AI for routine requests, we propose that automation of such services will occur in the form of an assisted model : chatbots will be deployed alongside a scaled back human labor force. This aligns with the CGS recommendation that companies should use AI solutions for quick service requests and allow remaining human agents to take more strategic roles and focus on detailed, complex customer inquiries [4]. Self-Service Begets Customer Anxiety Next, we discuss self-service tools and explore the effects of this type of interactive automation on customers. In a Harvard Business School study, Shell and Buell find that choices made in self-service settings are associated with customer anxiety that dampens satisfaction and undermines trust in the company [10]. Specifically, the absence of human contact in self-service transactions causes this ill sentiment. 20 Figure 12: The effect of access to human contact on choice satisfaction in high-anxiety conditions [10] The study consisted of participants completing an online investment task in which trad- ing decisions were made with and without access to human resources. Some participants were faced with a low anxiety trading environment marked by large returns on a healthy stock market while others operated in the high anxiety environment where the market (and returns) suffered. The study concluded that when granted access to human experts, partic- ipants in the high anxiety environment experienced a change in anxiety indistinguishable from that of participants in the low-anxiety treatment who were not granted human con- tact. In other words, access to human contact may be beneficial in self-service, high-anxiety contexts in its capacity to mitigate customer stress [10]. Moreover, access to human contact enhanced feelings of consumer satisfaction (Figure 12). The study continues by examining implementation strategies that use human labor to reduce customer anxiety most economically. They find that very few participants actually initiate contact and that the improvement in choice satisfaction only depends on having been given access to human contact. With this in mind, companies need not keep a costly labor force of service personnel but simply allow for potential access to live service. In other words, companies employing self-service tools or other interactive AI should employ the assisted model and scale back their human labor presence. Human Accountability and Emotional Concerns The study involving the use of LYNA in the tumor detection task actually employed and advocated for the assisted model in radiology tasks. Their advocacy of this model derives from a suite of concerns prominent in the field of medicine. Decisions made and actions taken can have profound repercus- sions on the livelihoods of the patients, so human accountability and emotional concerns must be taken into steeper consideration. LYNA does not output a diagnosis but a set of recommendations with which the radiologist can arrive at a conclusion. LYNA is tuned to 21 be hypersensitive–it would rather commit false positives and highlight healthy tissue than fail to identify tumors–so a radiologist must parse these false positives to determine the existence of real tumors [11]. Thus, the accuracy of the diagnosis is ultimately the respon- sibility of the radiologist. Should errors be made, the radiologist–not the algorithm–is held accountable. This helps provide credibility for the firm and assures high performance by the radiologists. Another challenge in the interactive segment of the task space pertains to the variability of emotional concerns surrounding the tasks. Diagnosis delivery and prescription assign- ment to patients with common colds or minor health problems are relatively low stress interactions. In contrast, telling a collegiate athlete their career is over due to injury or delivering news of a terminal illness to a patient and their family will involve intense, emo- tional interactions. In a 2019 CNN article, a family describes an unusual doctor visit. On the day before his death, Ernest Quintana was confronted by a robot mounted with a video display. The hospital attendant who had wheeled in the robot waited as a doctor displayed on the screen delivered his grim prognosis [8]. The patient’s granddaughter describes, “I think they should have had more dignity and treated him better than they did...We knew that we were going to lose him. Our point is the delivery (of the news). There was no compassion.” [8]. In this scenario, the prognosis was given by a human doctor not an AI alternative, but the use of an abnormal technological medium to deliver the news angered the family. This dissatisfaction stems not from the information exchanged but from the emotion surrounding the interaction. If the visit had been for a simple prescription renewal, expedited service performed by a robot would not have triggered this negative reaction. There is a space for technological tools in hospitals and other socially-demanding environments as long as the emotional stakes are low. In situations with high emotional stakes, AI solutions are ineffective because people demand human, face-to-face interaction. Partial Automation: The Assisted Model Through our discussion of chatbots in customer service, self-service technology and its effects on anxiety, and concerns of account- ability and emotional intelligence, we have demonstrated that automation of interactive tasks will occur in the form of the assisted-model. Routine requests in customer service calls, doctors visits, and self-checkout usage can and will be automated by AI tools. Com- panies must still maintain a human labor force, albeit decreased in footprint, to handle complex cases, to decrease consumer anxiety, and to provide accountability or emotional support in especially tense interactions. Having proposed and validated the assisted-model within this segment of the task space, we have completed our understanding of automation propensities across the entire task space. (Figure 13) 22 Figure 13: Automation propensities across the entire task space. 6 Conclusion Task-Based Automation Framework The prevalence of Deep Learning strategies in cutting edge artificial intelligence (AI) has significant implications for automation in the human labor market. To understand the scope of Deep Learning’s potential effect on the job market, we construct a framework with which we will analyze automation propensities. We first translate jobs into the aggregate of required tasks performed within this job and can then assess Deep Learning’s ability to complete each of these tasks. We consider a three-dimensional task space and analyze how AI performs along each of its three axes: the complexity, the degree of environmental rigidity, and the amount of social interaction required in the task. Tasks where AI performs well are at high-risk for automation while tasks where AI fails to achieve sufficient performance are at low risk for automation. With this foundation, we can slice jobs into the tasks that constitute them and can understand the job-level automation propensity in terms of the summation of its task-level automation propensities. As a starting point, we assume that simple tasks operating in static environments and requiring no social interaction (Simple /Mechanical / Non-Interactive) can be automated with existing technology. We support this assumption by demonstrating how such au- tomation is already integrated into society. In the remainder of the paper, we analyze how Deep Learning as a disruptive technology might extend AI’s automation capacity into other 23 segments of the task space. Deep Learning Nullifies Complexity Limitations With the status quo of AI capa- bility established, we explore how Deep Learning changes the picture. Through a demon- strative series of case studies, we determined the ability for Deep Learning solutions to handle increasing complexity in tasks. The only limit on the efficacy with which Deep Learning architectures handle high complexity is the quality and quantity of data on which these solutions are trained [7]. Deep Learning solutions can successfully complete financial time-series predictions despite the complexities surrounding stock market data [5]. LYNA, a Deep Learning-powered productivity tool, has proven indispensable in its capacity to im- prove radiologists’ tumor detection accuracy [11]. We conclude that with the breakthroughs in Deep Learning technology, AI can successfully operate at any degree of complexity. In other words, the computational complexity of a task has negligible effect on its propensity for automation. With this in mind, we collapse the task-space to two dimensions and focus on AI’s ability to operate in different types of environments and at varying degrees of social interaction. Deep Learning Struggles in Dynamic Environments Despite Deep Learning’s ad- vantages as a universally applicable device that can approximate functions of seemingly any complexity, the technology is decidedly ‘narrow’ AI [7]. With an adequate training dataset, a Deep Learning solution can complete specific tasks with sufficient efficacy. We tested, however, whether solution performance could be maintained when applied to tasks outside the one specifically trained on. Taking the game of Pacman as a trivial but demonstrative case study, we tested whether a Deep Learning controller learns fundamental evasion techniques that can be used in slightly altered Pacman maps [9]. Much as a human novice would quickly learn not to attempt to move through maze walls and to move toward regions with high food particle density, a solution that can endure the stresses of a dynamic environment would learn simi- lar fundamentals and could demonstrate semi-competent behavior when the task is slightly altered. The Deep Learning Pacman controller, however, moves randomly when introduced to novel map environments [12]. We then used the CartPole game to further show that per- formance of a Deep Learning model suffers when the environment parameters are adjusted even if the task difficulty remains the same [1]. These examples point to a defining trait of the technology: Deep Learning struggles to perform in environments outside those it was exposed to in training. A Deep Learning categorizer trained to distinguish between images containing or not containing a tumor would give a nonsensical response if tested on a sam- ple containing an alternate tissue anomaly. Deep Learning solutions require comprehensive training data and operate effectively only in ‘closed’ environments where the range of inputs is well established and an all-encompassing dataset can be constructed. We conclude that Deep Learning competency falls off as tasks environments become less rigidly singular and 24 uncertainties or variations are introduced. Partial Automation of Interactive Tasks: The Assisted Model Finally, we assess AI’s ability to operate interactively with the base assumption that AI solutions operate effectively in non-interactive scenarios. As a starting point, we investigated state-of-the- art chat bots in regards to Turing Test performance. Although judges can still effectively distinguish between human and bot-generated conversational text, we conclude that this disparity does not render chatbots useless. Consumers are willing to engage with chatbots for routine service requests even if they require live specialists for the more complex cases. [4]. We call this pattern of partial automation where AI tools perform some interactive tasks but do not entirely replace human personnel the assisted model and demonstrate its usefulness across other interactive tasks. Self-service technology can effectively be utilized to replace human labor in transactional tasks at the cost of increased consumer anxiety [10]. By employing the assisted model and providing access to human contact alongside self-service deployment, firms can dampen this anxiety [10]. In high stakes environments, especially in the field of medicine, deployment of AI solu- tions will be subject to issues of human accountability and emotional concerns. The steep consequences of tasks in these environments require actions to be rooted in human account- ability. Assistive tools like LYNA, a Deep Learning tool that highlights potential tumors, can be leveraged, but the diagnosis must ultimately be made by human radiologists [11]. Delivery of grim diagnoses necessitates oversight by a human doctor due to the complex emotions at play in such interactions [8]. When human accountability is required or intense emotions are at play, the assisted model will be employed; Deep Learning algorithms will serve as expert systems and humans will leverage this resource to deliver more accurate results but will rely on their own social skills to navigate emotional turbulence. Tasks requiring social interaction will be partially automated in the form of the assisted model with AI solutions being leveraged to reduce the workload and to amplify performance of human personnel. Areas at High Risk for Automation We conclude that the disruptive effects of Deep Learning architectures will have profound repercussions in areas of the job market defined by mechanical environments and static objectives. Though Deep Learning struggles in dynamic environments, its performance in mechanical tasks of any complexity provides what will be a much cheaper and more consistent source of labor for jobs with rigid rules and steady objectives. Financial planners, stock traders, and loan underwriters, for example, all complete tasks with static, well-defined objectives in relatively closed environments. This surrenders such jobs to a high risk of automation. Further, even jobs that require degrees of social interaction and emotional intelligence will feel the effects of automation. Under the assisted model, each worker will be more 25 efficient and accurate in their work with AI doing the more exhaustive, analytical work for them. When fewer but better-tooled employees can perform the same amount of work as the existing labor force, layoffs and downsizing are inevitable. With these results in mind, people should shape their academic choices and career paths to steer clear of the aforementioned high-risk zones. Emotional intelligence, social competency, and especially critical thinking in dynamic environments will be valuable skills because Deep Learning has a difficult time emulating human-levels of performance in these areas. References [1] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym, 2016. [2] Eugene Charniak. CSCI 1470 Introduction to Deep Learning. Brown University, 2018. [3] Roy De Kleijn, Marjolijn Wijnen, and Fenna Poletiek. The effect of context-dependent information and sentence constructions on perceived humanness of an agent in a turing test. Knowledge-Based Systems, 163:794–799, 2019. [4] Christopher Elliot. Chatbots are killing customer service. here’s why. Forbes, August 2018. [5] Christopher Fischer, Thomas; Krauss. Deep learning with long short-term memory networks for financial market prediction. FAU Discussion Papers in Economics, No. 11/2017, 2017. [6] Yann LeCun, Corinna Cortes, and Christopher JC Burges. The mnist database of handwritten digits, 1998. URL http://yann. lecun. com/exdb/mnist, 10:34, 1998. [7] K.F. Lee. AI Superpowers: China, Silicon Valley, and the New World Order. HMH Books, 2018. [8] Dakin Andone; Artemis Moshtaghian. A doctor in california appeared via video link to tell a patient he was going to die. the man’s family is upset. CNN, 03 2019. [9] P. Rohlfshagen, J. Liu, D. Perez-Liebana, and S. M. Lucas. Pac-manconquers academia: Two decades of research using a classic arcade game. IEEE Transactions on Games, 10(3):233–256, Sep. 2018. [10] Michelle A Shell and Ryan W Buell. Mitigating the negative effects of customer anxiety through access to human contact. Harvard Business School Technology & Operations Mgt. Unit Working Paper, 2019. 26 [11] David F Steiner, Robert MacDonald, Yun Liu, Peter Truszkowski, Jason D Hipp, Christopher Gammage, Florence Thng, Lily Peng, and Martin C Stumpe. Impact of deep learning assistance on the histopathologic review of lymph nodes for metastatic breast cancer. The American journal of surgical pathology, 42(12):1636–1646, 2018. [12] Tycho van der Ouderaa. Deep reinforcement learning in pac-man. University of Ams- terdam, 2016. 27