Papers is Alpha. This content is part of an effort to make research more accessible, and (most likely) has lost some details from the original. You can find the original paper here.
Introduction
AI-based software systems are rapidly becoming essential in many organizations. However, the focus on the technical side of building artificial intelligence (AI)-based systems are most common, and many projects, more often than not, fail to address critical human aspects during the development phases. These include but are not limited to age, gender, ethnicity, socio-economic status, education, language, culture, emotions, personality, and many others. Ignoring human-centered aspects in AI-based software tends to produce biased and non-inclusive outcomes. Shneidermanemphasizes the dangers of autonomy-first design in AI and the hidden biases that follow. Misrepresenting human aspects in requirements for model selection and data used in training AI algorithms can lead to discriminatory decision procedures even if the underlying computational processes were unbiased. For example, a study by Carnegie Mellon revealed that women were far less likely to receive high-paying job ads from Google than mendue to the under-representation of people of color and women in high paying IT jobs.
Studies on human-centered design aim to develop systems that put human needs and values at the center of software development and clearly understand the context of the software system’s usage. These human values include security, tradition, achievement, power, etc.. An increasing number of research resources are being invested in developing human-centered AI solutions. Large organizations such as Google, Microsoft, and Apple are moving towards including human-centered values when building AI-based software. These large organizations have now devised explicit guidelines for building human-centered AI systems. While these guidelines provide an excellent platform, they mainly focus on the design phase of software development and not on requirements engineering (RE).
Recent research efforts in RE have focused on including human-centered requirements in building software solutions, such as emotions,gender, power and politics, personality traits, age, and mental and physical challenges. For example, Miller et al.argued that “emotions should be considered as first-class citizens in software engineering methodology” and acknowledged the fact that software engineers usually overlook emotion when building software systems. Other studies have stressed that emotions are usually neglected when it comes to RE. Perera et al.emphasized the importance of including human values early on in RE.
Although this is a growing research area in RE, limited studies focus on including requirements for human-centered AI-based systems. We examined the studies presented in our systematic literature review (SLR)and a mapping studyon RE for AI (RE4AI) and found that most AI-based software lacks human-centered approaches when writing and modeling requirements, and existing research mainly focused on explainability, trust, and ethics with limited empirical evaluations.
We propose a new framework, the RE for human-centered AI (RE4HCAI), to specify and model requirements for human-centered AI-based software. The RE4HCAI framework is based on industrial human-centered guidelines, an analysis of studies obtained from the literature, and the results from an experts’ survey. The survey identified the guidelines and human-centered aspects that should be addressed and prioritized during RE. The framework is then evaluated in a case study, and requirements are elicited and modeled for a system that uses AI to enhance the quality of 360° videos for virtual reality (VR) users. The purpose of the AI-based software was to improve the quality of experience (QoE) and quality of service (QoS) for systems streaming and rendering 360° video contents. We discovered that some requirements could not be identified during the early stages of the project due to the black-box nature of the AI. The difficulty in explaining how it would respond to the available data also contributed to this issue. Therefore, a more iterative RE approach should be used to write requirements for AI-based software. The key contributions of this research can be described as follows:
We present a new framework to help elicit and specify requirements for AI-based software. The framework is based on the industry’s human-centered AI development, an SLR, and an expert survey. As part of our framework, we provide a catalog to aid the elicitation of requirements for AI-based software. The framework further provides a conceptual model to present the requirements visually.
We apply and evaluate the framework on a case study that uses AI to enhance the spatial quality of 360° VR videos and report on our findings.
The rest of the paper is structured as follows:Section sec-background provides a brief background related work. Section sec-framework presents details of our framework. Section sec-casestudy reports on implementing the framework and modeling language on a software system to enhance 360° videos for VR users. Section sec-discussion discusses key results and summarizes emerging theories. Section sec-threats addresses threats to validity. Section sec-relatedwork discusses related work, and section sec-conclusion concludes.
Background and Motivation
As mentioned in section sec-introduction, our framework is aimed at helping the elicitation and specification of requirements for human-centered AI-based software. We conducted an SLRand selected any existing study that focused on human-centered approaches in RE4AI. These papers include studies on emotion for a human-centered social robot, understating expectations and limitations of AI-based software, identifying and mitigating human-centered issues related to fairness and biases, ethics, explainability, and trust.Next, the industrial guidelines on human-centered AI development are analyzed, specifically Google’s PAIR guidebook,Apple’s human interface guidelines for building ML applications, Microsoft’s eighteen guidelines for human-centered AI interactionand the machine learning (ML) canvas. We note that ML canvas does not directly address the human-centered aspect. Nevertheless, we chose to include ML canvas for its relevance to our work as the ML canvas complemented the industrial guidelines by providing means for capturing relevant information for building ML models, e.g., ML business needs, decisions, data sources, and evaluation, and it facilitates collaboration between different stakeholders.
The gathered requirements from the SLR and the industrial guidelines are summarized, mapped, and categorized into six areas to build human-centered AI-based software. The categorization of six areas aligns with the five categories proposed in Google PAIR. The sixth area, i.e., model needs, identifies the human-centered approaches used when selecting and training an appropriate AI-based model. Each area is explained in detail below as follows:
Area#1 Requirements for User Needs
The first area focuses on ensuring that the user needs are captured first when building AI-based software, documenting the limitations and capabilities, what the system can do, how well it can do its assigned tasks, and identifying the users and stakeholders. Other factors include how the user will interact with the system and what approaches should be selected to match the user’s needs. Is the system going to be proactive or reactive? For example, in a proactive system, Google search provides a list of results based on the users’ entered keywords. The next step is to determine if the user is aware of the AI feature. Visible features are when the user is aware of the AI component, such as having auto-complete trying to guess what the user wants to input next. Invisible features are when the user is not aware of the AI feature. For example, a map would gather real-time data on traffic to provide the best route to users.
Next, we need to determine if the system is going to automate or augment the user’s needs. For instance, AI-based software can automate any task that does not require human oversight. Tasks that a user enjoys or requires a human-in-the-loop can be augmented by AI to improve the user’s experience and efficiency.
Finally, the evaluation behind the choice of reward function should be specified. How will the AI choose between right and wrong predictions of a learned model (LM)? An LM, as defined by Berry, “is the result of an instance of ML or deep learning (DL), whether the LM is taught, self-taught, or both with relevant real-world (RW) data”. In other words, an LM is a computational representation that is generated by an ML or DL algorithm. The model is learned by training on relevant RW data, which allows it to make predictions or decisions based on new, unseen data. The learning process typically involves feeding the algorithm a large dataset containing input-output pairs, from which the algorithm can discover patterns, relationships, or underlying structures in the data. During the training process, LM adjusts its internal parameters to minimize the error between its predictions and the actual outcomes. Once the model is trained and achieves a satisfactory level of performance, it can be applied to new, unseen data to make predictions, classify data points, or recommend actions, depending on the specific task it was designed to perform.
In the case of binary classification, a prediction by an LM can result in four categories, i.e., true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). The two reward functions available are precision and recall. Precision is the proportion of correctly picked TP out of the total positive predictions, i.e., precision = TP / (TP + FP). TP refers to the correctly predicted positive instances, while FP refers to instances where the LM predicts an incorrect positive outcome. A flight booking app with high precision would predict the cheapest flights for a specific date but might miss out on some flights. Recall measures the proportion of TP instances the LM can correctly identify. Thus, recall = TP / (TP + FN). Higher recall provides the confidence that the system has included all relevant results but might also include some irrelevant results. Evaluating the reward function will depend on the specific task and domain of the AI application used. The decision to use either would be based on the trade-off between selecting precision and recall. Therefore, when evaluating the reward function, a list of trade-offs should be documented to justify the selected function.
Area#2 Requirements for Model Needs
The second area includes documenting requirements for model needs. How do we choose an algorithm that optimizes stakeholder satisfaction? Do we need a system that is explainable or accurate? For example, Shin et al.look for ways to reduce cost by experimenting with different algorithms and found that some algorithms could produce better outcomes with lower costs. However, these algorithms might lack in other aspects, such as explainability. In contrast, different algorithms might provide a better explanation but with predictions with lower confidence. The choice of algorithm will also depend on the user’s needs for the system. Also, as part of requirement specifications for AI, there is the need to specify the possible settings of each variable for a given AI algorithm when applied to a particular task in a given context.
Part of the requirements specifications should consider how the incoming data affects model training. Thus, deciding how LM will improve on new data should be specified. Does the system need a dynamic LM that updates and trains online or a static system that improves only with updates? A dynamic system improves as the user interacts with the system and often involves using feedback in model training. A static system will improve with system updates and is trained offline. For example, an image recognition LM that depends on data that does not change over time will need updates with new releases, or else the LM might become biased or useless after a while.
Model selection depends on the choice of algorithm and can include supervised, unsupervised, and reinforcement learning. When training the system, we need to specify a threshold to avoid overfitting or underfitting the data and select tools that will be used to evaluate the LM. When tuning the LM, the following should be considered: types of feedback, user behavior, training data used in model tuning, and adjusting the parameters accordingly.
Area#3 Requirements for Data Needs
The third area focuses on data needs and data collection methods. Data collection includes the type and amount of data needed. Once data collection methods are specified, data requirements should be considered, including quality of data used, security, privacy settings, and fairness. Data quality includes five components: accuracy, completeness, consistency, credibility, and currentness. 1) Having correct data will ensure accuracy. 2) Completeness refers to the availability of all attributes and events associated with the data. 3) Consistency has no contradictions in the data used. 4) Ensuring credibility by having truthful data. 5) Finally, currentness means data must be collected within the correct time frame.
Data quantity focuses on data being diverse. And data quality addresses the completeness, consistency, and correctness of data being used. Selecting data should include identifying features, labels, and sampling rates. Labels identify the features needed to train the ML model, such as labeling a scanned image of a tumor as malignant. Explicit labeling is done manually, and implicit labeling is when the model learns the pattern independently. Examples represent a row of data and contain features, and labels represent descriptions given to data. More samples in the dataset ensure diversity but also increase costs. In this case, a threshold should be identified to set the amount of data needed within the given budget.
Identifying and reporting biases in data must be addressed. Such biases can include automation, selection, group attribution, etc. Automation biases are when preferences are selected based on automated suggestions from the system. Selection bias usually happens when data is not collected randomly from the target population but rather selected based on the stakeholder’s requests. Group attribution assumes that an output suitable for an individual will have the same impact on everyone in the group. Identifying key data characteristics should be set early to avoid discrimination and biases.
Area#4 Requirements for Feedback and User Control
This area deals with identifying and finding which kinds of user feedback need to be established in RE when building AI-based systems. Different types of feedback include implicit, explicit, and calibration. Implicit feedback provides information about the user’s interaction, preferences, and system behavior. Examples of implicit feedback include accepting or rejecting a recommendation, times of use, number of hours, when or how many times the user logged on, etc. The user provides explicit feedback when requested by the system, including surveys, forms, ratings, written feedback, likes, or dislikes. Calibration is the initial information the system might need from the user to function, such as scanning your fingerprint for the first time to activate touch ID.
It is essential to identify how and when the feedback will be used in model tuning and what changes it will have on the AI-based system. When asking for feedback, privacy measures should be considered to secure it. Also, give the user a choice to dismiss the feedback. Allowing users to feel that they control the system is essential to human-centered AI and can depend on many factors. Providing the users with control can be achieved by giving them complete control of the system in case of a failure or providing them with multiple options from which they can make the appropriate choice.
Area#5 Requirements for Explainability and Trust
AI-based systems reasoning can include explaining the results or predictions to the end user and stakeholders. From the stakeholders’ point of view, explainability requirements should consist of setting expectations of what the system can do and how it can do it and explaining its limitations and capabilities. On the other hand, explaining to the user will involve the data used and informing them about changes to the system that might happen with updates or LM improvements. Explaining predictions can be presented by either providing an example or displaying confidence. Confidence could be displayed in many ways depending on the situation and the AI-based system. Each situation should be evaluated to find an appropriate method to show confidence.
Explainable requirements can often conflict with others, such as performance and cost. It might be cheaper to build systems that are not very explainable with better performance. Therefore, it is vital to calculate the trade-off when favoring an explainable system and identify how it might conflict with other requirements. Schoonderwoerd et al.,indicate that explanations should be provided based on a specific context, with the need to identify which explanations should be provided and when. However, explainability might be necessary for some situations to ensure compliance with the European Union’s General Data Protection Regulation (GDPR). In this case, other measures need to be considered.
Providing realistic expectations helps users and stakeholders avoid over-trusting AI-based software. Explainability requirements might occur during the building process or after the LM is deployed. Explainability requirements can be divided into four components to include: Who the explanation is addressed to, what needs to be explained, when should the explanation happen, and who explains. Also, explanations should consist of consequences that might occur due to an action performed by the user. Showing the confidence can be a way to provide an explanation to users. However, it is essential to determine when and how to display predictions, as sometimes showing confidence could lead to mistrust.
Area#6 Requirements for Errors and Failure
None of the studies in the RE4AI literature mention the need to address errors in RE when building AI-based systems. The focus of most industrial guidelines is on designing AI-based software with a human-centered approach and does not focus on RE. Our survey resultsshow that practitioners working on AI-based systems wanted to know how to deal with errors and specify error sources during RE. The different types of errors include background, context, and system limitations. Some errors, such as background and context, are more challenging to identify and are invisible to the end user. Context errors happen for several reasons and can be avoided by ensuring the user understands how the system works and ensuring the system is aligned with their needs. They are usually an outcome that is a true positive yet does not provide a prediction that aligns with the user’s needs.
Error sources include system, incorrect predictions, data, and input and output errors. Data errors happen with mislabeled data and can be due to poor training or inaccurate labeling. Prediction errors can occur when an incorrect model is used, the data is not comprehensive, or missing some critical elements. Input errors happen when users input unexpected data and can be due to users’ old habits or having an abusive user. Output errors are when the system provides a prediction that is low in confidence or an irrelevant output but high in confidence. Moreover, finally, system errors would happen when multiple systems using AIintegrate or depend on each other. We need to identify error types and sources in RE and mitigate them by providing an action plan for how they should be addressed and fixed.
A Framework to Manage RE for Human-centered AI-based systems (RE4HCAI)
This section presents the RE4HCAI framework, illustrated in figure fig-re4hciframework, for eliciting and modeling requirements for human-centered AI-based systems. The framework consists of three layers. The first layer in section sec-referencemap presents the human-centered guidelines that should be included in RE. The guidelines are combined and mapped into a reference model. The second layer presents a catalog (provided in the appendix) to help elicit requirements for building human-centered AI-based software and is discussed in section sec-catalog. The last layer proposes a modeling language to present the requirements visually, as explained in section sec-modelinglanguage.
Human-centered AI Guidelines for RE
< g r a p h i c s >
Human-centered needs for AI and the frequency of survey participants who selected them.
The coverage of these six areas in RE was investigated by conducting a user survey with 29 practitioners and researchers working on AI-related projects. The participants included data scientists, ML specialists, and software engineers working with AI-based software. We asked each participant to identify which of the mapped human-centered guidelines are or should be, included in RE based on their experience. The survey results, shown in figure fig-needs, confirmed that all six human-centered areas need to be specified in RE4HCAI. The survey participants responded more favorably towards including the first three areas of user, model, and data needs than the remaining three areas for RE4HCAI. Thus, our evaluation focuses on these three areas in section sec-casestudy.
The six areas of the human-centered AI components that are needed to be considered in RE are mapped to present a reference model that showcases the overall layout of the framework, as shown in figure fig-framework. Each area presented in the reference model is explained further in section sec-background.
Catalog for Collecting Requirements
This section presents a catalog of human-centered AI requirements along the six areas in our framework. The gathered human-centered requirements and the mapped requirements from the SLR are listed in a tabular format to form our catalog. The catalog can be used as a checklist to elicit requirements for AI-based software and has six sections. Each section is dedicated to eliciting detailed requirements for each of the six areas explained in the background section sec-background. The catalog is provided in the appendix.
Modeling Requirements for Human-centered AI
In our SLR, we sought any existing modeling language or requirements notations used in RE4AI. UML was the most popular method, making it easier for non-software engineers to work with and learn. However, UML has limitations as it does not support modeling non-functional requirements (NFR) and business rules. Goal-oriented requirements engineering (GORE) had better support for NFR and business rules but was more challenging to learn and was mainly used by requirements engineers and software engineers. Also, GORE could model requirements at lower levels of abstraction than UML. Silva et al.explained that using GORE to model a requirement or concept could be presented with fewer structural diagrams than UML. However, UML is more widely known and used than GORE. Also, GORE was reported to be more challenging to learn among non-software engineers.
Looking at the different roles from the survey, we found that around 25% of the people involved in building AI-based software were software engineers and requirements engineers. Data scientists and ML specialists contributed around 27% of the team building AI-based software. Other roles included system and business analysts, developers, and researchers, as shown in figure fig-roles. Due to the diverse nature of team structure in building AI-based software and the need to use more specific modeling concepts to reflect on the process provided in the framework, we decided to create a conceptual model for the RE4HCAI framework, as conceptual modeling allows for precise concepts to be modeled as well as presenting a holistic view of the application.
Our conceptual model consists of two levels. The first level presented in figure fig-metamodel provides a holistic view of the system requirements. We use an oval shape notation to list the main goal that needs to be addressed. For example, in the case study we conduct in section sec-casestudy, the goal would be enhancing the quality of 360°videos by four times. This goal would connect to the six sub-models to include the six areas of our framework, areas#1-6 covered in section sec-background. For each area, we use a UML class diagram to present it. Each area contains the main attributes in the reference model in figure fig-framework. The second level consists of a sub-model to show each area in further detail. We provide the notations to model these requirements in table table-legend. When building the visual notations for our language, we attempt to comply with the nine principles of notationsto reduce the complexity and cognitive load for visual notations in SE modeling. We present our notations using different shapes, textures, icons, and colors.
Goals are presented using an oval shape; each need includes the components modeled in the sub-models. Since our framework focuses mostly on needs, we use a square with dash lines to visually emphasize the concept of needs. Elongated hexagons are used to present processes or tasks. Trade-offs are displayed using an equilateral octagon. Limitations and capabilities are modeled using rectangles to show what the AI component can and cannot do, with limitations shaded in yellow to differentiate between them visually. Icons were used to present some notations.The folder icon is used for attributes, the stick icon for users, and the data icon for data sources.Moreover, finally, the components are joined using connectors to either show the flow of data or evaluate the data.
When presenting the models, we use blue tick icons on notations to represent the parts that belong to the framework. The notations that do not include a blue tick icon are system specific, as shown in figure fig-sysvsarch. System requirements will change when modeling a different AI-based system.
Case Study Application and Evaluation
This section describes the process we used to design, select, and conduct the case study to implement our proposed framework. The case study answers the following research questions (RQs):
How does our RE4HCAI framework benefit the process of eliciting and modeling requirements for human-centered AI software? RQ1 aims to assess the usefulness of our framework in eliciting and modeling requirements. It further contrasts the RE process before and after implementing our framework in the case study.
How can the RE practices be aligned in the life cycle of AI projects? RQ2 aims to provide an understanding and alignment of the RE practices in AI-based software development projects.
The case study follows the guidelines for conducting case study researchand empirical evaluations in software engineering. The main reason for conducting this study was to find how to extract requirements for building human-centered AI-based software and if implementing the framework benefited the process. The catalog is used to elicit requirements for the AI system, to identify user needs, model needs, and data needs as presented in tables table-userneeds, table-modelneeds, and table-dataneeds. These requirements are then modeled, as shown in figures fig-userneeds, fig-modelneeds, and fig-dataneeds.
Study selection
When selecting the case studies, we had the following selection criteria in mind:
Software project with a significant AI component: Since our framework focuses on RE4AI, we required a project with an AI component to answer our RQs, and, ideally, a project that involved interaction with users.
Project maturity: The project should be at a reasonably mature stage of development but ideally still under development. Our previous experience motivated this criterion, as we found that experts tend to miss out on details in already completed projects. Furthermore, to address RQ2, we required a project in which the requirements had been established using traditional methods.
Availability and background of the expert: We needed to select a project with experts that had the requisite knowledge of the AI component of the system so we could elicit and model the requirements for model and data needs. Also, the case study required substantial time commitment from the expert(s), and hence we sought projects in which at least one expert agreed to the time commitment.
Our case study on VR-360° video enhancer matched the abovementioned criteria. The VR project builds on a DL model to enhance the quality of 360° videos for VR platforms. Hence, it has a major AI component that satisfies the first criteria. Also, the project meets the second selection criteria as it was at the later stages of development. The project’s requirements were finalized as a software feature plan, and most of the features were implemented when we conducted the case study. Furthermore, we had access to the primary ML expert of the project, who developed the AI component and agreed on the time commitment, which satisfies the third selection criteria.
Data Collection
We collected data over four sessions with the ML expert. Of these four sessions, three were dedicated to extracting and modeling the requirements for the project, and a fourth session evaluated the modeled human-centered requirements. The ML expert who built the AI project, and ended up being the fourth author, had no prior background knowledge of how our framework or the modeling language worked. They were given a brief overview of our framework in the first session. During the fourth session, we identified which requirements could be established at the start of the project. In other words, these were the requirements that the ML expert found important for them to know at the start of the project’s lifecycle. However, some of these requirements were not necessarily known originally by the experts and were explored through our framework.
We note that we could elicit and model the requirements only for the first three areas of the framework collaboratively. This was the case due to the limited availability of the ML expert and the fact that the elicitation and modeling for each area required significant time commitment. While we ideally wanted to cover all six areas, we had a choice of covering a subset thoroughly or covering all areas superficially. We made the former choice, which aligns with our survey findings that practitioners’ first three areas are the most focused areas when eliciting requirements.
In the first session, the ML expert gave an overview of the project, and the expert was given a brief overview of the motivation of the study. The expert was further given a brief overview of the framework. The first session was dedicated to collaboratively eliciting and then modeling the first area for user needs. The first session was the longest and lasted $\approx$ 4 hours and was divided into two periods. In the first half, the requirements were elicited, and the second half consisted of modeling the requirements. The second session was dedicated to the second area model needs. The session took $\approx$ 2.5 hours and was done over two periods. Similar to the first session, the requirements were elicited in the first half, followed by modeling the requirements in the second half. In line with the first two sessions, the third session was dedicated to eliciting and modeling the third area data needs. The session took $\approx$ 3 hours over two periods. The fourth session took two hours, and we evaluated all three models. We instructed the ML expert that they were free to suggest any changes to the elicited and modeled requirements from the three sessions. Also, the ML expert reflected on the comparison between the RE process followed originally in the project and the requirements collected from the sessions after applying our framework. The holistic view of the system is presented in figure fig-level1.
Improving the Quality of 360° Videos
The case study was conducted on a project that uses DL to enhance the quality of 360° videos for VR platforms. Current approaches to building 360° videos can capture and facilitate the immersive and interactive viewing experience. However, the quality of the final product can degrade due to the limitations of consumer-grade hardware, bandwidth streaming constraints, and processing that requires stitching videos from multiple cameras. The proposed solution was to enhance the quality of the final product using AI-based software.
Requirements for User Needs
The first session was held to elicit requirements for identifying the user’s needs. The session started by identifying the need for the proposed AI-based software using the catalog in the appendix. The need for the system was determined by the quality of the existing 360° videos. Most 360° videos had low resolution and contained artifacts and noise, thus affecting the quality of the final product. The process of removing unwanted noise and artifacts was not achievable without the AI component, therefore, establishing the need for an AI-based software.
Identified user needs requirements for the 360video enhancer.* requirements that could be specified before testing the data on the selected model
Table Label: table-userneeds
Download PDF to view table
To identify the system’s capabilities, we needed to understand what the expected system could and could not do. System capabilities covered both the end-user and stakeholders. From the user’s perspective, the end product would improve the resolution four times while using the same hardware equipment. Therefore, improving the QoE in either entertainment or educational settings. The limitations affected both the end-user and the stakeholders. Listing these limitations to avoid high expectations from either side was important. The limitations for the end-user were that users had to own VR equipment capable of rendering high-quality 360° videos. The second limitation to the user was that they could watch these videos only offline. Therefore, online streaming would not be possible. Furthermore, the limitation for the stakeholder was towards building the system, which included hardware resources and time needed to process the videos. The stakeholders needed to establish ways to improve quality without increasing expenses, which will later link to model selection and collected data.
When modeling limitations and challenges, we found that using color helped distinguish between them visually, as shown in figure fig-userneeds. We modeled the human-centered aspects as goals, and each goal would be mapped to either a need, process, capability, or limitation. For example, the system needed to be automated because it was not possible for a human to do the system’s task. Processes involved actions that needed to be implemented, such as building a reactive system that doesn’t require user interaction. Also, having different textures for the needs made it easier to identify them visually. The advantage of modeling the first area was that it was easy to recognize limitations, capabilities, and needs when building human-centered AI-based software.
Requirements for Model Needs
The second session involved collecting requirements for model needs from our catalog, which helped us understand the rationale behind selecting the type of algorithm used to train the LM and other aspects considered during optimization. The LM was built using deep learning techniques and as a regression problem and optimized for accuracy in order to produce better-quality frames. Model tuning included adjusting the LM to user feedback, user behavior, and output errors. For this particular case, user feedback would be done with a ranking scale that is given after the user watches the enhanced video. The other form of feedback would be to perform a user study to evaluate the quality of the generated outcome from the model. The study would involve human subjects ranking the quality on a scale of 1 to 100. This feedback will indicate if the human-centered perceptual quality of the predicted outcome is acceptable. If not, the plan would be to change the model training and optimize it to improvise perceptual quality. We note that we modeled as limitations the cases where user or system goals could not be achieved, e.g., tracing LM output errors back to data.
Identified model needs requirements for the 360 video enhancer.* Requirements that could be specified before testing the data on the selected model
Table Label: table-modelneeds
Download PDF to view table
Trade-offs that needed to be calculated and modeled were connected to scaling the LM. Scaling was achieved by changing the width or depth in the deep learning model. Increasing the depth meant adding more convolution layers and allowing the model to learn more complex patterns. The depth would have to increase if the performance were not good enough. However, the trade-off to increasing the depth is that more computational resources are required, resulting in delays in the output generation. On the contrary, changing the width would require that features extracted from the input become wider. This would require more memory allocation for data, as it would store more features for the given input at multiple layers. For the model used, an experiment would be set to identify when to modify width or depth and at what intervals while evaluating the trade-offs for time vs. costs to achieve the most effective results. Table table-modelneeds shows the collected requirements for model needs.
Identified data needs requirements for the 360 video enhancer.* Requirements that could be specified before testing the data on the selected model
Table Label: table-dataneeds
Download PDF to view tableRequirements for Data Needs
The third session involved extracting requirements for data needs. While conducting this session, it was found that the data collected originally was unsuitable for the model’s training. Therefore, the data had to be cleaned and adjusted to become accurate, correct, consistent, and current. Sudden changes in scene, motion, or light and removing rolling and credit information were removed to make the data accurate. Next, to ensure data completeness, a list of all attributes and events needed for model training was listed, and the expert ensured each attribute had data to represent. Completeness was established technically to show the diversity of the presented data to ensure that the dataset had diversity in terms of 1) motion,2) content, which included objects and scenes, and 3) light. The initially collected data was not consistent in quality. Thus, videos were processed to the same length and resolution to ensure consistency. Finally, currentness was provided by creating video shots representing a unique single scene, thus ensuring that each clip/shot had current data for a continuous period.
Data requirements presented a higher number of needs. The need to specify how the data will be selected and collected for use in model training and testing. For data selection, the needs consisted of setting features, labels, examples, and sampling rates. Features include, among others, color, size, and location. An essential aspect of data needs was to identify any biases that might happen due to the data used. The proposed system was designed for videos with insignificant motion, content, and light changes across time. However, the model might perform differently if the videos had larger motion or luminance changes. Moreover, the diversity of the scenes and objects found was limited to what was available in the open-sourced platforms creating the possibility of the same scene/object represented across multiple videos. Thus, more diverse scenes would be needed to avoid under or over-representing data.
Data collection involved gathering diverse content, including different objects such as trees, buildings, humans, and animals, and various motions such as how objects move across the video and lighting situations such as day, night, bright light, and dim. These measures were identified before data collection and obtained from literature based on the International Telecommunication Union’s standard guidelines for spatio-temporal complexity measures. Although measures were taken into account, some constraints were found after training the model with the collected dataset. The constraints were related to image quality and size due to hardware and model memory-related limitations. These constraints were modeled as limitations to show that data was limited to the specified size and frame rate.
Discussion
This section reflects on how applying our framework helped write and model requirements for AI-based systems.
RQ1. How does our RE4HCAI framework benefit the process of eliciting and modeling requirements for human-centered AI software?
Most traditional RE methods are not equipped to manage AI-based software, and requirements are sometimes difficult to write, especially during the early stages of systems development. The problem with specifying requirements for an AI-based software project is that it is difficult to explain the black-box nature of AI and how the system would work or what outcome it will predict. In our case study, we found that the process of eliciting and specifying requirements for AI-based software is vastly different from traditional approaches, as some aspects of the system are unpredictable and difficult to explain.
To evaluate the framework and modeling tool, we compared the building process of the 360° VR model before and after the implementation of our framework. Before implementing the framework, many human-centered aspects were missing from the final product. Most of the development was based on the technical aspects of building the AI-based system. For example, classifying what data was needed was based on technical aspects (e.g., model and hardware needs) rather than focusing on the human perspective. After applying the framework, we identified possible biases that might lead to issues, e.g., biases related to the lack of diversity of data sources and the biases related to the lack of diversity of the type of videos.
The catalog helped elicit most of the requirements to build the AI-based software project from a human-centered perspective. The models presented in figures fig-userneeds, fig-modelneeds, and fig-dataneeds helped the ML expert visualize what needed to be considered when implementing the AI-based software. The conceptual models for our RE4HCAI framework provided a visual view of all the components in the system. Our participant said “It helped me visualize all the components needed to be done in my project”. Also, we found that limitations and system needs were easier to spot with the visual representation via the conceptual model. Another observation we found missing was using feedback in the LM. The initial configuration of the LM did not consider the users’ view of how good the perceptual quality of the improved 360° videos was. However, after applying the framework, the decision was made to include user feedback to account for model tuning in future work of the 360° VR project.
We also noticed that some framework aspects would change depending on the application domain and the AI type used. For example, in this case study, regression was used, and the optimized reward function would be the loss function. Therefore, we need to consider other requirements that might need to be elicited and specified in our framework. This relates to Berry’s findingthat different AI algorithms and their individual settings in different application contexts would require different requirements specifications. However, if the system were designed by applying a classification algorithm and we had to choose between precision or recall, one would need to calculate the trade-offs. When calculating the trade-off between recall vs. precision, we need to list the implications of having an FP vs. FN. In this case, if precision is favored, i.e., we would opt to reduce the number of FPs at the cost of tolerating more FNs, then the 360° VR system will use the correctly selected TP frames but remove some frames from the outcome FN. Thus, the final outcome will have visual gaps, as some relevant frames will be missing. On the other hand, favoring recall meant that most frames would be included, i.e., more FPs. Thus, the final output will also include frames that are not enhanced FP, compromising the quality of experience as the visual quality will not always be the same in the final product, and some of the frames will still have the same quality as the original video.
The downside to favoring precision would be the possibility of having lags when rendering frames, as the system might choose to exclude frames that should have been enhanced from the frame sequence. These lags would increase the chance of latency which is a potential cause of VR sickness. Decreasing this delay or latency would result in reduced sickness. Recall would still include these frames when rendering, even if these frames might not have the expected quality. In this case, the trade-off is to compromise the visual quality instead of causing lags or delays in the visual display.
RQ2. How can the RE practices be aligned in the life cycle of AI projects?
When applying the framework to the case study, not all requirements could be elicited at the start of the project. We found that it was important to identify some requirements at the start. For example, when identifying user needs, we found that it was necessary to have initial specifications before building AI-based software. Also, some of the capabilities could be identified only after getting initial results from the model training, such as how much it would improve the quality of experience and if it would affect the quality of service. Furthermore, limitations such as hardware resources and processing time could not be identified at the start of the project. The extent of the limitation was found after training the model with the existing dataset. The reason was that the ML expert could not know how much scaling the model needed. Adding more features and layers resulted in higher processing time and required more resources to accommodate the change. They could establish how many features or layers were needed to improve model performance only after several training iterations.
Similar patterns were found in both requirements for model needs and data needs. We observed that these requirements would change over time as they learned how the model would interact with the available data. For example, information regarding the needed frames could be specified only after initial testing. The datasets were limited to using up to 20 frames per clip, which was unknown at the beginning of the project. However, after initial testing, it was found that this was a constraint due to the model and hardware limitations. The same applied to identifying the quantity and diversity of the data. After experimenting on the first round of training and testing, they found that the existing 360 dataset was inadequate. Thus, the dataset had to be modified accordingly.
We established that it was important to specify requirements for data needs, such as data sources, charges, diversity of collected data, data quality, if data was up to date, and other requirements involving cleaning the data to provide accurate and current results. Some of these data requirements were obtained from literature and standard guidelines. Therefore, not all requirements could be specified at the start of the project, and some might change or appear while model testing is in progress. We highlight the requirements that we could identify prior to starting the project in tables table-userneeds, table-modelneeds, and table-dataneeds by adding a * character before the requirement.
Threats to Validity
In all the phases of our research method and case study design, we attempted to mitigate and reduce any threat to validity as follows :
Internal Validity:
Case studies are easier to perform than experiments., but more challenging to interpret. The main disadvantage to conducting case studies is that the outcomes are more susceptible to researcher bias, and evaluation usually depends on how the researchers interpret the results. To reduce potential threats and biases, we ensured when selecting the case study that the person leading the AI project had no prior knowledge of our framework. Also, we collected requirements based on our framework and compared the results to how it was built without applying the proposed framework.
Construct Validity:
Another threat was the selection of studies to build the catalog. The risks of building a catalog for requirements are that it might miss out on some requirements and not be comprehensive. We established our requirements for the framework based on the guidelines and the literature presented on human-centered AI. Although we might have missed out on some requirements, we did base our findings on our SLR, which covered a comprehensive list of studies on RE4AI research. Also, we used industrial guidelines such as Google and Microsoft, which have already done extensive research to provide their guidelines.
External Validity:
The framework has been applied to only one case study, which threatens external validity. We will address this threat by conducting more case studies in future work and investigating how the framework applies to projects from different application domains and multiple stages in the software development lifecycle. Also, due to time limitations, we could only assess the first three areas of the framework and plan to conduct a further evaluation to assess the last three areas in future work.
Related Work
AI-based software should be carefully assessed not to replace people’s abilities but augment their capabilities and allow people to make the final choice. Shneidermanexplained the importance of enabling humans to control the AI-based system. Human needs and values in building AI-based software need to be researched and examined carefully. For example, AI bots such as Alexia and Siri require extensive training to obtain the right personality, and some companies are investing in creating algorithms for chatbots that can detect sarcasm or respond with empathy.
Recent studies have shown that many AI-based systems lack requirements specifications, which is mainly due to the difference in the building process between traditional systems vs. AI-based software. We conducted an SLRto identify literature on RE4AI. From the results, we identified existing frameworks in RE4AI to include. Nalchigar et al.offers a framework to manage RE in building ML systems. The presented GR4ML framework covered three views: the business view, the analytic design view, and the data preparation view. A conceptual modeling language was used to present each view visually. The framework was applied to a case study that investigated the use of ML in the medical domain.
Most AI-based systems are guided by large amounts of data and limited given resources. Bosch et al.proposed the Holistic DevOps framework for building AI-based software. The framework combined three practices: requirements-driven, data-driven, and AI-driven software systems. Requirements-driven approaches are used for systems that don’t require frequent changes. This method was found to use fewer resources when testing the system. On the other hand, data-driven approaches are used for systems that are constantly changing and updatedand are designed based on the analysis of available data. Companies that usually use automation, such as speech and image recognition, tend to use AI-driven approaches.
Aydemir and Dalpiazproposed a framework to aid requirements engineers and stakeholders in analyzing ethical requirements. The framework assisted in extracting, managing, and evaluating ethical requirements. However, the focus was on ethical requirements only and did not include other aspects of human-centered AI requirements. The last frameworkprovides a virtual framework to test specified requirements for a self-driving car. Also, frameworks are built to manage aspects of AI-bases systems, such as Khalajzadeh et al.who created a domain-specific modeling tool to support data analytic solutions for ML systems (BiDaML). The toolset provides five diagrams to support a different aspect of big data analytics.
Shneidermanpresented a framework that proposes 15 recommendations for engineering trustworthy, reliable, and safe human-centered AI-based systems. To ensure reliable human-centered AI, software engineering teams need to apply technically sound practices such as using appropriate analysis tools, updating workflows for each task and domain, applying new forms of validation and verification, testing for bias detection, and providing explainable user interfaces. For reliable human-centered AI, there should be more commitment and training to ensure safety measures, reporting, and addressing errors and failures. And lastly, promoting trustworthy systems by providing independent oversight reviews to support legal and ethical codes of conduct. Another method used to aid in creating awareness of ethics when building AI software is the ECCOLAmethod, which is based on the ethical AI guidelines and involves using a deck of 21 cards, with each card targeting an aspect of ethics.
Although some frameworks are proposed to provide methods for managing RE4AI, none have focused on delivering an overall solution to building human-centered AI in RE. An exception is Shneiderman’s frameworkthat relies on promoting human-centered AI-based software. However, it focuses on the entire Software Engineering process, not RE. This calls for researching and studying appropriate human-centered AI methods before including them in software systems. We argue that the proposed solutions in RE should address human needs, as having a very well-engineered product would not be useful if it does not satisfy the user’s needs.
Conclusion & Future Work
This paper offers a framework to extract and model requirements for human-centered AI. The proposed framework consists of three layers. The first layer provides a reference map to six areas from the human-centered AI guidelines. The second layer presents a catalog to elicit these requirements. The third layer provides a modeling tool to show the second layer’s elicited requirements visually. The framework was applied to a case study, and we extracted requirements for the first three layers of user, model, and data needs. The case included specifying and modeling requirements for an AI system that enhanced the quality of 360° VR videos. We found that some requirements were more difficult to specify initially and could be identified only after testing the model with the existing data.
We plan further to evaluate our framework in the future in several workshops and compare it with existing frameworks and modeling platforms. Also, we do not provide details of how attributes are connected in sub-level models. For example, in model needs, when modeling the goal for balancing overfitting and underfitting, we did not model the actual process. This would have to be presented in a separate sub-model. We plan to extend our model to present this level of detail in future work.
Appendix
1.2 p3cm p6cm p6cm
Catalog presenting requirements needed for Human-centered AI
3lc]@l@User Needs
29emCan AI add value or solve the problem to user’s needs? Do you have the expertise to manage AI? (Data scientist, ML specialist, SE) IF no – use a non-AI solution
Is data available? (format – amount – diverse)
3*Identify need for AI? Who are the users
Why do we need the system
What is the system used for
49emWhat are the system capabilities?
2*Limitations Limitations to the user:
Limitations to stakeholders:
Capabilities (What can the system do)
How well can the system do what it does?
Interaction with user action Proactive vs. Reactive (Provides results when people request them) or Reactive (Interacts with the user without requesting)
Is the user aware of AI feature? Visible or Invisible features
Evaluate approach Augmentation vs Automation (user enjoyment, control, high stakes) vs (boring, dangerous, cannot be performed by humans)
89emReward function or evaluation matrix Impact of having a false positive
Impact of having a false negative
Trade off - e.g precision vs recall Precision (excludes relevant results, removes all false positives, and misses some of the true positive predictions) vs recall (includes irrelevant results as it includes all true positive but might also include some false positive.)
Choice of reward function e.g precision or recall
Impact on user
List potential pitfalls
How do you provide inclusion
Predictions (How are predictions used in making decisions)
3*Predictions How are predictions used in making decisions
Impact of new input on predictions
Impact of feedback on predictions
3lc]@l@Model Needs
Optimize algorithm for? Optimize for accuracy, explainability, robust, etc.
Choose the ML type Supervised , Unsupervised, Reinforcement learning
Model training Static vs Dynamic (Offline evaluation / Training improves with updates) vs (Learns from user behavior, Improves from feedback)
2lc]@l@Balance between overfitting & underfitting
510emModel Tuning (Include parameter tuning and architecture changes) User feedback
Adjusting to user behavior
Trace output errors to data
Parameter tuning
Quantitative feedback
2lc]@l@Specify scalability issues
2lc]@l@Choose tools to use to evaluate the model
2lc]@l@ Evaluate the quality of the model
3lc]@l@Data Needs
4*Data selection Features (a characteristics of an input variable.E.g. Feature would include color, size, location, etc)
Labels descriptions given to data (Explicit -manually vs Implicit – model learns)
Example (a row of data and contains features and labels)
Sampling rate amount of data in a given dataset
39emData collection (Match data to user needs) Identify type of data needed
Amount of data needed
Diversity of data
Constraints cost, accuracy, quality, time?
3*Data source Type of data source (public, private, mixed)
Is data responsibly sourced?
Using feedback as data
Charges in obtaining data
Measures taken to ensure data is up to date?
2*Split Data Training data (Model tuning)
Testing data
5*Data quality Accuracy (The correctness of the data collected)
Completeness (all attributes and events should have data to associate them).
-List all attributes and events needed for the AI product -Make sure each attributes has data to represent
Consistency (Collected data has to be free from contradictions)
Credibility (Credible data had to be authentic and truthful)
Currentness (corresponded to having data collected within the correct time frame; for example, when collecting images of people for a facial recognition application, a picture of an adult person when they were a baby will not be the correct data)
2*Data requirements Protect personal information
Does data comply with privacy and law
4*Fairness Identify biases
Missing features
Under or over representative data
Other
5*Reporting biases Automation bias Automation biases are when preferences are selected based on automated suggestions from the system
Selection bias Selection bias usually happens when data is not collected randomly from the target population but rather selected based on the stakeholder's requests
Impact bias Making an assumption based on the persons judgment (e.g. all wedding dresses are white)
Group attribution Group attribution assumes that an output suitable for an individual will have the same impact on everyone in the group
Other
Address biases What methods will be used?
3lc]@l@Feedback & Control
69emExplicit feedback (Only ask when needed) What reward will be provided to the end-user or raters to provide explicit feedback (Material / symbolic)
If the reward is symbolic What impact will it have on the user? What benefit will the user gain from it?
How is it going to be reviewed and evaluated?
What changes will it make to the AI model?
How will it be used in model tuning?
How is feedback going to be conducted (surveys, notifications, ratings, etc.)
3*Implicit feedback User interactions / behaviours to use: Frequency of use, time of use, amount of time spent interacting with the system, accept / reject recommendations, etc.
Review feedback from user interaction to how it will make changes to AI
How will it be used in model tuning
3*Calibration Reason for using calibration
When should the user use calibration
How is it going to affect the AI
49emWhen asking for feedback Measures taken to secure user privacy
Provide multiple options to feedback (what options will you provide?)
Is feedback in line with user mental map or users understanding of the system
Allow the user to dismiss feedback or opt-out (How are you going to provide this option?)
4*User control When to give the user control (High stakes, legal, safety )
Allow user to adjust preferences
Allow user dismissal
Level of control (high, medium, low or no control)
3lc]@l@Explainability & Trust
2*Explain to stakeholders
Limitations of AI
Capabilities of AI
2*Explain to user Inform user when changes happen
Explain consequences to users action
3*Explain data: Who can access or use the user’s personal data
How data is shared between apps
Explain how predictions are based on data
3*Explain predictions Explain with examples
Explain with confidence: Data visualization (Expert users).Numerical (Might cause confusion).Categorial (e.g., Low - Med - High).N-best Alternatives (Low confidence)
Do not display Confidence: Risks (misleading, no impact to user decisions when explaining output, distracting).Low confidence (Can provide partial explanation
Calculate Trade-off Identify conflicts with other requirements
3lc]@l@Calibrate user trust
2*Explain special cases
Law & rules
Third party involvement
2*Explain feedback Impact of feedback on AI
How feedback is used to improve model
3*Avoid over trusting
Only show relevant info
Account for different situations -Low stakes.High stakes
Explain errors
3lc]@l@Errors & Failures
39emUser perceived errors (might change over time) Failstates True negative (Not included in system training).System limitation
Context errors (these are true positive but) AI making incorrect assumptions.User has poor mental model.System not aligned with users needs
Background errors Invisible errors the user cannot perceive
139emList possible errors source
Prediction & data errors
Mislabeled or misclassified results Poor training.Incorrect labeling
Incorrect Model
Incomplete data
**Input Errors**
Unexpected input
Old habits
Mis-calibrated input
**Output Quality**
Low confidence Lack of data. Uncertain prediction accuracy. Unstable information
**systems hierarchy error**
Multiple systems (Allow user to give one of the system priorities)
Crashing signals (Allow signals from the primary system only)
2*Error risks High stakes Health, safety and financial decisions -Sensitive / private data
Abusive users
3*Action Mitigate Errors
Allow users to fix mistakes
Provide suggestions when in doubt
Bibliography
1@article{milne2012power,
2 publisher = {Springer},
3 year = {2012},
4 pages = {83--98},
5 volume = {17},
6 journal = {Requirements Engineering},
7 author = {Milne, Alastair and Maiden, Neil},
8 title = {Power and politics in requirements engineering: embracing the dark side?},
9}
10
11@article{callele2008emotional,
12 publisher = {IEEE},
13 year = {2008},
14 pages = {43--45},
15 number = {1},
16 volume = {25},
17 journal = {IEEE software},
18 author = {Callele, David and Neufeld, Eric and Schneider, Kevin},
19 title = {Emotional requirements},
20}
21
22@inproceedings{thompson2013evoking,
23 year = {2013},
24 booktitle = {Looking into the Future of Creativity and Decision Support Systems: Proceedings of the 8th International Conference on Knowledge, Information and Creativity Support Systems},
25 author = {Thompson, Clare and Maiden, Neil and Nouri, Mobina and Zachos, Konstantinos},
26 title = {Evoking emotion through stories in creative dementia care},
27}
28
29@article{maiden2012spocks,
30 publisher = {IEEE Computer Society},
31 year = {2012},
32 pages = {84--85},
33 number = {04},
34 volume = {29},
35 journal = {IEEE software},
36 author = {Maiden, Neil},
37 title = {Spocks and kirks in the requirements universe},
38}
39
40@article{ramos2005emotion,
41 publisher = {Springer},
42 year = {2005},
43 pages = {238--242},
44 volume = {10},
45 journal = {Requirements Engineering},
46 author = {Ramos, Isabel and Berry, Daniel M},
47 title = {Is emotion relevant to requirements engineering?},
48}
49
50@article{agrahari2022omnidirectional,
51 publisher = {TechRxiv},
52 year = {2022},
53 author = {Agrahari Baniya, Arbind and Lee, Glory and Eklund, Peter and Aryal, Sunil},
54 title = {Omnidirectional Video Super-Resolution using Deep Learning},
55}
56
57@article{vakkuri2021eccola,
58 publisher = {Elsevier},
59 year = {2021},
60 pages = {111067},
61 volume = {182},
62 journal = {Journal of Systems and Software},
63 author = {Vakkuri, Ville and Kemell, Kai-Kristian and Jantunen, Marianna and Halme, Erika and Abrahamsson, Pekka},
64 title = {ECCOLA—A method for implementing ethically aligned AI systems},
65}
66
67@inproceedings{Zamani:21,
68 doi = {10.1109/REW53955.2021.00023},
69 pages = {116-125},
70 number = {},
71 volume = {},
72 year = {2021},
73 title = {Machine Learning in Requirements Engineering: A Mapping Study},
74 booktitle = {2021 IEEE 29th International Requirements Engineering Conference Workshops (REW)},
75 author = {Zamani, Kareshna and Zowghi, Didar and Arora, Chetan},
76}
77
78@inproceedings{khalajzadeh2020visual,
79 year = {2020},
80 pages = {15--26},
81 booktitle = {ENASE},
82 author = {Khalajzadeh, Hourieh and Simmons, Andrew J and Abdelrazek, Mohamed and Grundy, John and Hosking, John G and He, Qiang},
83 title = {Visual Languages for Supporting Big Data Analytics Development.},
84}
85
86@inproceedings{khalajzadeh2019bidaml,
87 organization = {IEEE},
88 year = {2019},
89 pages = {93--97},
90 booktitle = {2019 IEEE International Congress on Big Data (BigDataCongress)},
91 author = {Khalajzadeh, Hourieh and Abdelrazek, Mohamed and Grundy, John and Hosking, John and He, Qiang},
92 title = {Bidaml: A suite of visual languages for supporting end-user data analytics},
93}
94
95@article{wachter2017counterfactual,
96 publisher = {HeinOnline},
97 year = {2017},
98 pages = {841},
99 volume = {31},
100 journal = {Harv. JL \& Tech.},
101 author = {Wachter, Sandra and Mittelstadt, Brent and Russell, Chris},
102 title = {Counterfactual explanations without opening the black box: Automated decisions and the GDPR},
103}
104
105@article{goodman2017european,
106 year = {2017},
107 pages = {50--57},
108 number = {3},
109 volume = {38},
110 journal = {AI magazine},
111 author = {Goodman, Bryce and Flaxman, Seth},
112 title = {European Union regulations on algorithmic decision-making and a “right to explanation”},
113}
114
115@article{shneiderman2020bridging,
116 publisher = {ACM New York, NY, USA},
117 year = {2020},
118 pages = {1--31},
119 number = {4},
120 volume = {10},
121 journal = {ACM Transactions on Interactive Intelligent Systems (TiiS)},
122 author = {Shneiderman, Ben},
123 title = {Bridging the gap between ethics and practice: guidelines for reliable, safe, and trustworthy human-centered AI systems},
124}
125
126@article{lloyd2018bias,
127 year = {2018},
128 journal = {arXiv preprint arXiv:1809.07842},
129 author = {Lloyd, Kirsten},
130 title = {Bias amplification in artificial intelligence systems},
131}
132
133@article{itu1999subjective,
134 year = {1999},
135 author = {ITU-T RECOMMENDATION, P},
136 title = {Subjective video quality assessment methods for multimedia applications},
137}
138
139@incollection{calders2013unbiased,
140 publisher = {Springer},
141 year = {2013},
142 pages = {43--57},
143 booktitle = {Discrimination and privacy in the information society},
144 author = {Calders, Toon and {\v{Z}}liobait{\.e}, Indr{\.e}},
145 title = {Why unbiased computational processes can lead to discriminative decision procedures},
146}
147
148@article{shneiderman2020Three,
149 year = {2020},
150 pages = {109--124},
151 number = {3},
152 volume = {12},
153 journal = {AIS Transactions on Human-Computer Interaction},
154 author = {Shneiderman, Ben},
155 title = {Human-centered artificial intelligence: three fresh ideas},
156}
157
158@inproceedings{grundy2021impact,
159 year = {2021},
160 pages = {9--20},
161 booktitle = {ENASE},
162 author = {Grundy, John C},
163 title = {Impact of End User Human Aspects on Software Engineering.},
164}
165
166@inproceedings{mcintosh2021evaluating,
167 organization = {IEEE},
168 year = {2021},
169 pages = {31--40},
170 booktitle = {2021 IEEE/ACM 13th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE)},
171 author = {McIntosh, Jennifer and Du, Xiaojiao and Wu, Zexian and Truong, Giahuy and Ly, Quang and How, Richard and Viswanathan, Sriram and Kanij, Tanjila},
172 title = {Evaluating Age Bias In E-commerce},
173}
174
175@book{shneiderman2022human,
176 publisher = {Oxford University Press},
177 year = {2022},
178 author = {Shneiderman, Ben},
179 title = {Human-Centered AI},
180}
181
182@article{schwartz2012overview,
183 year = {2012},
184 pages = {2307--0919},
185 number = {1},
186 volume = {2},
187 journal = {Online readings in Psychology and Culture},
188 author = {Schwartz, Shalom H},
189 title = {An overview of the Schwartz theory of basic values},
190}
191
192@inproceedings{riedl2016using,
193 year = {2016},
194 booktitle = {Workshops at the Thirtieth AAAI Conference on Artificial Intelligence},
195 author = {Riedl, Mark O and Harrison, Brent},
196 title = {Using stories to teach human values to artificial agents},
197}
198
199@article{whittle2019your,
200 publisher = {IEEE},
201 year = {2019},
202 pages = {112--115},
203 number = {3},
204 volume = {36},
205 journal = {IEEE Software},
206 author = {Whittle, Jon},
207 title = {Is your software valueless?},
208}
209
210@article{maguire2001methods,
211 publisher = {Elsevier},
212 year = {2001},
213 pages = {587--634},
214 number = {4},
215 volume = {55},
216 journal = {International journal of human-computer studies},
217 author = {Maguire, Martin},
218 title = {Methods to support human-centred design},
219}
220
221@inproceedings{schmidt2020interactive,
222 year = {2020},
223 pages = {1--4},
224 booktitle = {Proceedings of the International Conference on Advanced Visual Interfaces},
225 author = {Schmidt, Albrecht},
226 title = {Interactive human centered artificial intelligence: a definition and research challenges},
227}
228
229@article{bellamy2018ai,
230 year = {2018},
231 journal = {arXiv preprint arXiv:1810.01943},
232 author = {Bellamy, Rachel KE and Dey, Kuntal and Hind, Michael and Hoffman, Samuel C and Houde, Stephanie and Kannan, Kalapriya and Lohia, Pranay and Martino, Jacquelyn and Mehta, Sameep and Mojsilovic, Aleksandra and others},
233 title = {AI Fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias},
234}
235
236@inproceedings{roselli2019managing,
237 year = {2019},
238 pages = {539--544},
239 booktitle = {Companion Proceedings of The 2019 World Wide Web Conference},
240 author = {Roselli, Drew and Matthews, Jeanna and Talagala, Nisha},
241 title = {Managing bias in AI},
242}
243
244@article{whittaker2019disability,
245 year = {2019},
246 journal = {AI Now Institute},
247 author = {Whittaker, Meredith and Alper, Meryl and Bennett, Cynthia L and Hendren, Sara and Kaziunas, Liz and Mills, Mara and Morris, Meredith Ringel and Rankin, Joy and Rogers, Emily and Salas, Marcel and others},
248 title = {Disability, bias, and AI},
249}
250
251@article{dignum2017responsible,
252 publisher = {Daffodil International University},
253 year = {2017},
254 author = {Dignum, Virginia},
255 title = {Responsible artificial intelligence: designing AI for human values},
256}
257
258@article{hussain2020human,
259 publisher = {IEEE},
260 year = {2020},
261 journal = {IEEE Transactions on Software Engineering},
262 author = {Hussain, Waqar and Perera, Harsha and Whittle, Jon and Nurwidyantoro, Arif and Hoda, Rashina and Shams, Rifat Ara and Oliver, Gillian},
263 title = {Human values in software engineering: Contrasting case studies of practice},
264}
265
266@inproceedings{perera2020study,
267 organization = {IEEE},
268 year = {2020},
269 pages = {409--420},
270 booktitle = {2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE)},
271 author = {Perera, Harsha and Hussain, Waqar and Whittle, Jon and Nurwidyantoro, Arif and Mougouei, Davoud and Shams, Rifat Ara and Oliver, Gillian},
272 title = {A study on the prevalence of human values in software engineering publications, 2015-2018},
273}
274
275@inproceedings{viyovic2014sirius,
276 organization = {IEEE},
277 year = {2014},
278 pages = {233--238},
279 booktitle = {IEEE 18th International Conference on Intelligent Engineering Systems INES 2014},
280 author = {Viyovi{\'c}, Vladimir and Maksimovi{\'c}, Mirjam and Perisi{\'c}, Branko},
281 title = {Sirius: A rapid development of DSM graphical editor},
282}
283
284@article{moody2009physics,
285 publisher = {IEEE},
286 year = {2009},
287 pages = {756--779},
288 number = {6},
289 volume = {35},
290 journal = {IEEE Transactions on software engineering},
291 author = {Moody, Daniel},
292 title = {The “physics” of notations: toward a scientific basis for constructing visual notations in software engineering},
293}
294
295@article{kitchenham1995case,
296 publisher = {IEEE},
297 year = {1995},
298 pages = {52--62},
299 number = {4},
300 volume = {12},
301 journal = {IEEE software},
302 author = {Kitchenham, Barbara and Pickard, Lesley and Pfleeger, Shari Lawrence},
303 title = {Case studies for method and tool evaluation},
304}
305
306@incollection{bosch2021engineering,
307 publisher = {IGI global},
308 year = {2021},
309 pages = {1--19},
310 booktitle = {Artificial Intelligence Paradigms for Smart Cyber-Physical Systems},
311 author = {Bosch, Jan and Olsson, Helena Holmstr{\"o}m and Crnkovic, Ivica},
312 title = {Engineering ai systems: A research agenda},
313}
314
315@article{glymour2008causal,
316 publisher = {Lippincott Williams \& Wilkins Philadelphia, PA},
317 year = {2008},
318 pages = {183--209},
319 volume = {3},
320 journal = {Modern epidemiology},
321 author = {Glymour, M Maria and Greenland, Sander},
322 title = {Causal diagrams},
323}
324
325@article{pearl1995causal,
326 publisher = {Oxford University Press},
327 year = {1995},
328 pages = {669--688},
329 number = {4},
330 volume = {82},
331 journal = {Biometrika},
332 author = {Pearl, Judea},
333 title = {Causal diagrams for empirical research},
334}
335
336@article{greenland1999causal,
337 publisher = {JSTOR},
338 year = {1999},
339 pages = {37--48},
340 journal = {Epidemiology},
341 author = {Greenland, Sander and Pearl, Judea and Robins, James M},
342 title = {Causal diagrams for epidemiologic research},
343}
344
345@inproceedings{blumreiter2019towards,
346 organization = {IEEE},
347 year = {2019},
348 pages = {543--548},
349 booktitle = {2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)},
350 author = {Blumreiter, Mathias and Greenyer, Joel and Garcia, Francisco Javier Chiyah and Kl{\"o}s, Verena and Schwammberger, Maike and Sommer, Christoph and Vogelsang, Andreas and Wortmann, Andreas},
351 title = {Towards self-explainable cyber-physical systems},
352}
353
354@book{embley2012handbook,
355 publisher = {Springer},
356 year = {2012},
357 author = {Embley, David W and Thalheim, Bernhard},
358 title = {Handbook of conceptual modeling: theory, practice, and research challenges},
359}
360
361@article{martinez2021software,
362 year = {2021},
363 journal = {arXiv preprint arXiv:2105.01984},
364 author = {Mart{\'\i}nez-Fern{\'a}ndez, Silverio and Bogner, Justus and Franch, Xavier and Oriol, Marc and Siebert, Julien and Trendowicz, Adam and Vollmer, Anna Maria and Wagner, Stefan},
365 title = {Software Engineering for AI-Based Systems: A Survey},
366}
367
368@inproceedings{zhou2016map,
369 organization = {IEEE},
370 year = {2016},
371 pages = {153--160},
372 booktitle = {2016 23rd Asia-Pacific Software Engineering Conference (APSEC)},
373 author = {Zhou, Xin and Jin, Yuqin and Zhang, He and Li, Shanshan and Huang, Xin},
374 title = {A map of threats to validity of systematic literature reviews in software engineering},
375}
376
377@article{holmquist2017intelligence,
378 publisher = {ACM},
379 year = {2017},
380 pages = {28--33},
381 number = {4},
382 volume = {24},
383 journal = {interactions},
384 author = {Holmquist, Lars Erik},
385 title = {Intelligence on tap: artificial intelligence as a new design material},
386}
387
388@article{price2016microsoft,
389 year = {2016},
390 journal = {Business Insider},
391 author = {Price, Rob},
392 title = {Microsoft is deleting its \uppercase{AI} chatbot’s incredibly racist tweets},
393}
394
395@article{khomh2018software,
396 publisher = {IEEE},
397 year = {2018},
398 pages = {81--84},
399 number = {5},
400 volume = {35},
401 journal = {IEEE Software},
402 author = {Khomh, Foutse and Adams, Bram and Cheng, Jinghui and Fokaefs, Marios and Antoniol, Giuliano},
403 title = {Software Engineering for Machine-Learning Applications: The Road Ahead},
404}
405
406@inproceedings{houdek2017automotive,
407 year = {2017},
408 booktitle = {REFSQ Workshops},
409 author = {Houdek, Frank and Schmerler, Stefan},
410 title = {Automotive Future and its Impact on Requirements Engineering.},
411}
412
413@incollection{boy2014unifying,
414 publisher = {Springer},
415 year = {2014},
416 pages = {151--162},
417 booktitle = {Complex Systems Design \& Management},
418 author = {Boy, Guy A and Narkevicius, Jennifer McGovern},
419 title = {Unifying human centered design and systems engineering for human systems integration},
420}
421
422@book{li2017artificial,
423 publisher = {CRC press},
424 year = {2017},
425 author = {Li, Deyi and Du, Yi},
426 title = {Artificial intelligence with uncertainty},
427}
428
429@inproceedings{zowghi2002three,
430 year = {2002},
431 pages = {155--164},
432 booktitle = {International Workshop on Requirements Engineering: Foundations for Software Quality, Essen, Germany: Essener Informatik Beitiage},
433 author = {Zowghi, Didar and Gervasi, Vincenzo},
434 title = {The Three {C}s of requirements: consistency, completeness, and correctness},
435}
436
437@article{jiang2017artificial,
438 publisher = {BMJ Specialist Journals},
439 year = {2017},
440 pages = {230--243},
441 number = {4},
442 volume = {2},
443 journal = {Stroke and vascular neurology},
444 author = {Jiang, Fei and Jiang, Yong and Zhi, Hui and Dong, Yi and Li, Hao and Ma, Sufeng and Wang, Yilong and Dong, Qiang and Shen, Haipeng and Wang, Yongjun},
445 title = {Artificial intelligence in healthcare: past, present and future},
446}
447
448@article{washizaki2019studying,
449 year = {2019},
450 journal = {arXiv preprint arXiv:1910.04736},
451 author = {Washizaki, Hironori and Uchida, Hiromu and Khomh, Foutse and Gueheneuc, Yann-Gael},
452 title = {Studying Software Engineering Patterns for Designing Machine Learning Systems},
453}
454
455@inproceedings{kanij2015empirical,
456 organization = {IEEE Press},
457 year = {2015},
458 pages = {1--7},
459 booktitle = {Proceedings of the Eighth International Workshop on Cooperative and Human Aspects of Software Engineering},
460 author = {Kanij, Tanjila and Merkel, Robert and Grundy, John},
461 title = {An empirical investigation of personality traits of software testers},
462}
463
464@article{soomro2016effect,
465 publisher = {Elsevier},
466 year = {2016},
467 pages = {52--65},
468 volume = {73},
469 journal = {Information and Software Technology},
470 author = {Soomro, Arjumand Bano and Salleh, Norsaremah and Mendes, Emilia and Grundy, John and Burch, Giles and Nordin, Azlin},
471 title = {The effect of software engineers’ personality traits on team climate and performance: A Systematic Literature Review},
472}
473
474@incollection{kamalrudin2014pair,
475 publisher = {Springer},
476 year = {2014},
477 pages = {150--164},
478 booktitle = {Requirements Engineering},
479 author = {Kamalrudin, Massila and Sidek, Safiah and Salleh, Norsaremah and Hosking, John and Grundy, John},
480 title = {A Pair-Oriented Requirements Engineering Approach for Analysing Multi-lingual Requirements},
481}
482
483@article{miller2015emotion,
484 publisher = {Elsevier},
485 year = {2015},
486 pages = {54--71},
487 volume = {105},
488 journal = {Journal of Systems and Software},
489 author = {Miller, Tim and Pedell, Sonja and Lopez-Lorca, Antonio A and Mendoza, Antonette and Sterling, Leon and Keirnan, Alen},
490 title = {Emotion-led modelling for people-oriented requirements engineering: The case study of emergency systems},
491}
492
493@article{miller2012understanding,
494 publisher = {Elsevier},
495 year = {2012},
496 pages = {2160--2170},
497 number = {9},
498 volume = {85},
499 journal = {Journal of Systems and Software},
500 author = {Miller, Tim and Pedell, Sonja and Sterling, Leon and Vetere, Frank and Howard, Steve},
501 title = {Understanding socially oriented roles and goals through motivational modelling},
502}
503
504@inproceedings{burnett2016finding,
505 organization = {ACM},
506 year = {2016},
507 pages = {2586--2598},
508 booktitle = {Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems},
509 author = {Burnett, Margaret and Peters, Anicia and Hill, Charles and Elarief, Noha},
510 title = {Finding gender-inclusiveness software issues with {G}ender{M}ag: a field investigation},
511}
512
513@article{kissoon2019emotion,
514 publisher = {Elsevier},
515 year = {2019},
516 pages = {215--229},
517 volume = {147},
518 journal = {Journal of systems and software},
519 author = {Kissoon Curumsing, Maheswaree and Fernando, Niroshinie and Abdelrazek, Mohamed and Vasa, Rajesh and Mouzakis, Kon and Grundy, John},
520 title = {Emotion-oriented requirements engineering: a case study in developing a smart home system for the elderly},
521}
522
523@book{dick2017requirements,
524 publisher = {Springer},
525 year = {2017},
526 author = {Dick, Jeremy and Hull, Elizabeth and Jackson, Ken},
527 title = {Requirements engineering},
528}
529
530@inproceedings{vorvoreanu2019From,
531 year = {2019},
532 booktitle = {Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI’19). ACM, New York, NY, USA. https://doi. org/10.1145/3290605.3300283},
533 author = {Vorvoreanu, Mihaela and Zhang, Lingyi and Huang, Yun-Han and Hilderbrand, Claudia and Steine-Hanson, Zoe and Burnett, Margaret},
534 title = {From Gender Biases to Gender-Inclusive Design: An Empirical Investigation},
535}
536
537@article{grundy2018supporting,
538 year = {2018},
539 pages = {75--90},
540 volume = {246},
541 journal = {Studies in health technology and informatics},
542 author = {Grundy, John and Mouzakis, Kon and Vasa, Rajesh and Cain, Andrew and Curumsing, Maheswaree and Abdelrazek, Mohamed and Fernando, Niroshine},
543 title = {Supporting Diverse Challenges of Ageing with Digital Enhanced Living Solutions.},
544}
545
546@inproceedings{sneed2007testing,
547 organization = {IEEE},
548 year = {2007},
549 pages = {380--387},
550 booktitle = {Seventh International Conference on Quality Software (QSIC 2007)},
551 author = {Sneed, Harry M},
552 title = {Testing against natural language requirements},
553}
554
555@article{graham2002requirements,
556 publisher = {IEEE},
557 year = {2002},
558 pages = {15--17},
559 number = {5},
560 volume = {19},
561 journal = {IEEE Software},
562 author = {Graham, Dorothy},
563 title = {Requirements and testing: Seven missing-link myths},
564}
565
566@inproceedings{sneed2018requirement,
567 organization = {Springer},
568 year = {2018},
569 pages = {60--79},
570 booktitle = {International Conference on Software Quality},
571 author = {Sneed, Harry M},
572 title = {Requirement-based testing-extracting logical test cases from requirement documents},
573}
574
575@inproceedings{vorvoreanu2019rom,
576 year = {2019},
577 booktitle = {Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI’19). ACM, New York, NY, USA. https://doi. org/10.1145/3290605.3300283},
578 author = {Vorvoreanu, Mihaela and Zhang, Lingyi and Huang, Yun-Han and Hilderbrand, Claudia and Steine-Hanson, Zoe and Burnett, Margaret},
579 title = {from Gender Biases to Gender-Inclusive Design: An Empirical Investigation},
580}
581
582@inproceedings{amershi2019software,
583 organization = {IEEE Press},
584 year = {2019},
585 pages = {291--300},
586 booktitle = {Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice},
587 author = {Amershi, Saleema and Begel, Andrew and Bird, Christian and DeLine, Robert and Gall, Harald and Kamar, Ece and Nagappan, Nachiappan and Nushi, Besmira and Zimmermann, Thomas},
588 title = {Software engineering for machine learning: a case study},
589}
590
591@article{inayat2015systematic,
592 publisher = {Elsevier},
593 year = {2015},
594 pages = {915--929},
595 volume = {51},
596 journal = {Computers in human behavior},
597 author = {Inayat, Irum and Salim, Siti Salwah and Marczak, Sabrina and Daneva, Maya and Shamshirband, Shahaboddin},
598 title = {A systematic literature review on agile requirements engineering practices and challenges},
599}
600
601@article{wilson2018collaborative,
602 year = {2018},
603 pages = {114--123},
604 number = {4},
605 volume = {96},
606 journal = {Harvard Business Review},
607 author = {Wilson, H James and Daugherty, Paul R},
608 title = {Collaborative intelligence: humans and {AI} are joining forces},
609}
610
611@article{bellamy2019think,
612 publisher = {IEEE},
613 year = {2019},
614 pages = {76--80},
615 number = {4},
616 volume = {36},
617 journal = {IEEE Software},
618 author = {Bellamy, Rachel KE and Dey, Kuntal and Hind, Michael and Hoffman, Samuel C and Houde, Stephanie and Kannan, Kalapriya and Lohia, Pranay and Mehta, Sameep and Mojsilovic, Aleksandra and Nagar, Seema and others},
619 title = {Think your artificial intelligence software is fair? Think again},
620}
621
622@inproceedings{sculley2015hidden,
623 year = {2015},
624 pages = {2503--2511},
625 booktitle = {Advances in neural information processing systems},
626 author = {Sculley, David and Holt, Gary and Golovin, Daniel and Davydov, Eugene and Phillips, Todd and Ebner, Dietmar and Chaudhary, Vinay and Young, Michael and Crespo, Jean-Francois and Dennison, Dan},
627 title = {Hidden technical debt in machine learning systems},
628}
629
630@incollection{maalej2013introduction,
631 publisher = {Springer},
632 year = {2013},
633 pages = {1--20},
634 booktitle = {Managing Requirements Knowledge},
635 author = {Maalej, Walid and Thurimella, Anil Kumar},
636 title = {An introduction to requirements knowledge},
637}
638
639@incollection{fricker2015requirements,
640 publisher = {Springer},
641 year = {2015},
642 pages = {25--46},
643 booktitle = {Requirements Engineering for Digital Health},
644 author = {Fricker, Samuel A and Grau, Rainer and Zwingli, Adrian},
645 title = {Requirements engineering: best practice},
646}
647
648@book{koelsch2016requirements,
649 publisher = {Springer},
650 year = {2016},
651 author = {Koelsch, George},
652 title = {Requirements writing for system engineering},
653}
654
655@book{dean2003managing,
656 publisher = {Addison-Wesley Professional},
657 year = {2003},
658 author = {Dean, Leffingwell and Don, Widrig},
659 title = {Managing software requirements: A use case approach},
660}
661
662@book{marshall2009solid,
663 publisher = {Microsoft Press},
664 year = {2009},
665 author = {Marshall, Donis and Bruno, John},
666 title = {Solid Code: Optimizing the Software Development Life Cycle},
667}
668
669@book{mcconnell1998software,
670 publisher = {Pearson Education},
671 year = {1998},
672 author = {McConnell, Steve},
673 title = {Software project survival guide},
674}
675
676@article{horkoff2019goal,
677 publisher = {Springer},
678 year = {2019},
679 pages = {133--160},
680 number = {2},
681 volume = {24},
682 journal = {Requirements Engineering},
683 author = {Horkoff, Jennifer and Aydemir, Fatma Ba{\c{s}}ak and Cardoso, Evellin and Li, Tong and Mat{\'e}, Alejandro and Paja, Elda and Salnitri, Mattia and Piras, Luca and Mylopoulos, John and Giorgini, Paolo},
684 title = {Goal-oriented requirements engineering: an extended systematic mapping study},
685}
686
687@inproceedings{bano2014systematic,
688 organization = {IEEE},
689 year = {2014},
690 pages = {9--16},
691 booktitle = {2014 IEEE 4th International Workshop on Empirical Requirements Engineering (EmpiRE)},
692 author = {Bano, Muneera and Zowghi, Didar and Ikram, Naveed},
693 title = {Systematic reviews in requirements engineering: A tertiary study},
694}
695
696@article{amyot2010evaluating,
697 publisher = {Wiley Online Library},
698 year = {2010},
699 pages = {841--877},
700 number = {8},
701 volume = {25},
702 journal = {International Journal of Intelligent Systems},
703 author = {Amyot, Daniel and Ghanavati, Sepideh and Horkoff, Jennifer and Mussbacher, Gunter and Peyton, Liam and Yu, Eric},
704 title = {Evaluating goal models within the goal-oriented requirement language},
705}
706
707@article{daneva2014empirical,
708 publisher = {Elsevier},
709 year = {2014},
710 pages = {1--9},
711 volume = {95},
712 journal = {Journal of systems and software},
713 author = {Daneva, Maya and Damian, Daniela and Marchetto, Alessandro and Pastor, Oscar},
714 title = {Empirical research methodologies and studies in Requirements Engineering: How far did we come?},
715}
716
717@inproceedings{lwakatare2019taxonomy,
718 organization = {Springer},
719 year = {2019},
720 pages = {227--243},
721 booktitle = {International Conference on Agile Software Development},
722 author = {Lwakatare, Lucy Ellen and Raj, Aiswarya and Bosch, Jan and Olsson, Helena Holmstr{\"o}m and Crnkovic, Ivica},
723 title = {A taxonomy of software engineering challenges for machine learning systems: An empirical investigation},
724}
725
726@article{ur2013analysis,
727 year = {2013},
728 pages = {40},
729 number = {3},
730 volume = {5},
731 journal = {International Journal of Information Technology and Computer Science (IJITCS)},
732 author = {ur Rehman, Tousif and Khan, Muhammad Naeem Ahmed and Riaz, Naveed},
733 title = {Analysis of requirement engineering processes, tools/techniques and methodologies},
734}
735
736@inproceedings{van2004goal,
737 organization = {IEEE},
738 year = {2004},
739 pages = {4--7},
740 booktitle = {Proceedings. 12th IEEE International Requirements Engineering Conference, 2004.},
741 author = {Van Lamsweerde, Axel},
742 title = {Goal-oriented requirements enginering: a roundtrip from research to practice [enginering read engineering]},
743}
744
745@article{amyot2011user,
746 year = {2011},
747 pages = {747--768},
748 number = {5},
749 volume = {6},
750 journal = {JSW},
751 author = {Amyot, Daniel and Mussbacher, Gunter},
752 title = {User requirements notation: the first ten years, the next ten years},
753}
754
755@inproceedings{belani2019requirements,
756 organization = {IEEE},
757 year = {2019},
758 pages = {252--255},
759 booktitle = {2019 IEEE 27th International Requirements Engineering Conference Workshops (REW)},
760 author = {Belani, Hrvoje and Vukovic, Marin and Car, {\v{Z}}eljka},
761 title = {Requirements Engineering Challenges in Building {AI}-Based Complex Systems},
762}
763
764@inproceedings{arpteg2018software,
765 organization = {IEEE},
766 year = {2018},
767 pages = {50--59},
768 booktitle = {2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)},
769 author = {Arpteg, Anders and Brinne, Bj{\"o}rn and Crnkovic-Friis, Luka and Bosch, Jan},
770 title = {Software engineering challenges of deep learning},
771}
772
773@article{lovejoy2018human,
774 year = {2020. https://medium.com/google-design/human-centered-machine-learning-a770d10562cd},
775 journal = {Medium. Retrieved March},
776 author = {Lovejoy, Josh and Holbrook, Jess},
777 title = {Human-Centered Machine Learning. 7 steps to stay focused on the user when designing with {ML}},
778}
779
780@inproceedings{amershi2019guidelines,
781 year = {2019},
782 pages = {1--13},
783 booktitle = {Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems},
784 author = {Amershi, Saleema and Weld, Dan and Vorvoreanu, Mihaela and Fourney, Adam and Nushi, Besmira and Collisson, Penny and Suh, Jina and Iqbal, Shamsi and Bennett, Paul N and Inkpen, Kori and others},
785 title = {Guidelines for human-{AI} interaction},
786}
787
788@article{kitchenham2007guidelines,
789 publisher = {Citeseer},
790 year = {2007},
791 author = {Kitchenham, Barbara and Charters, Stuart},
792 title = {Guidelines for performing systematic literature reviews in software engineering},
793}
794
795@article{miller2017explainable,
796 year = {2017},
797 journal = {IJCAI 2017 Workshop on Explainable Artificial Intelligence (XAI), 36–42, URL http://people. eng.unimelb.edu.au/tmiller/pubs/explanation-inmates.pdf},
798 author = {Miller, Tim and Howe, Piers and Sonenberg, Liz},
799 title = {Explainable {AI}: Beware of inmates running the asylum or: How I learnt to stop worrying and love the social and behavioural sciences},
800}
801
802@inproceedings{sakamoto2013goal,
803 organization = {IEEE},
804 year = {2013},
805 pages = {32--35},
806 booktitle = {2013 3rd International Workshop on Games and Software Engineering: Engineering Computer Games to Enable Positive, Progressive Change (GAS)},
807 author = {Sakamoto, Kazunori and Hosono, Hiroaki and Sato, Seiji and Washizaki, Hironori and Fukazawa, Yoshiaki},
808 title = {Goal-oriented requirements analysis and an extended design pattern using scala for artificial intelligence programming contests},
809}
810
811@incollection{vassev2014autonomy,
812 publisher = {Springer},
813 year = {2014},
814 pages = {105--172},
815 booktitle = {Autonomy Requirements Engineering for Space Missions},
816 author = {Vassev, Emil and Hinchey, Mike},
817 title = {Autonomy requirements engineering},
818}
819
820@inproceedings{kondermann2013ground,
821 year = {2013},
822 pages = {1--4},
823 booktitle = {Proceedings of the International Workshop on Video and Image Ground Truth in Computer Vision Applications},
824 author = {Kondermann, Daniel},
825 title = {Ground truth design principles: an overview},
826}
827
828@inproceedings{hajian2016algorithmic,
829 year = {2016},
830 pages = {2125--2126},
831 booktitle = {Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining},
832 author = {Hajian, Sara and Bonchi, Francesco and Castillo, Carlos},
833 title = {Algorithmic bias: From discrimination discovery to fairness-aware data mining},
834}
835
836@article{morandini2017engineering,
837 publisher = {Springer},
838 year = {2017},
839 pages = {77--103},
840 number = {1},
841 volume = {22},
842 journal = {Requirements Engineering},
843 author = {Morandini, Mirko and Penserini, Loris and Perini, Anna and Marchetto, Alessandro},
844 title = {Engineering requirements for adaptive systems},
845}
846
847@inproceedings{dimatteo2020requirements,
848 year = {2020},
849 booktitle = {REFSQ Workshops},
850 author = {DiMatteo, Johnathan and Berry, Daniel M and Czarnecki, Krzysztof},
851 title = {Requirements for Monitoring Inattention of the Responsible Human in an Autonomous Vehicle: The Recall and Precision Tradeoff.},
852}
853
854@inproceedings{wickramasinghe2020trustworthy,
855 organization = {IEEE},
856 year = {2020},
857 pages = {130--136},
858 booktitle = {2020 13th International Conference on Human System Interaction (HSI)},
859 author = {Wickramasinghe, Chathurika S and Marino, Daniel L and Grandio, Javier and Manic, Milos},
860 title = {Trustworthy {AI} Development Guidelines for Human System Interaction},
861}
862
863@inproceedings{krause2016interacting,
864 year = {2016},
865 pages = {5686--5697},
866 booktitle = {Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems},
867 author = {Krause, Josua and Perer, Adam and Ng, Kenney},
868 title = {Interacting with predictions: Visual inspection of black-box machine learning models},
869}
870
871@article{miller2019explanation,
872 publisher = {Elsevier},
873 year = {2019},
874 pages = {1--38},
875 volume = {267},
876 journal = {Artificial Intelligence},
877 author = {Miller, Tim},
878 title = {Explanation in artificial intelligence: Insights from the social sciences},
879}
880
881@inproceedings{de2017people,
882 year = {2017},
883 booktitle = {2017 AAAI Fall Symposium Series},
884 author = {De Graaf, Maartje MA and Malle, Bertram F},
885 title = {How people explain action (and autonomous intelligent systems should too)},
886}
887
888@article{jobin2019global,
889 publisher = {Nature Publishing Group},
890 year = {2019},
891 pages = {389--399},
892 number = {9},
893 volume = {1},
894 journal = {Nature Machine Intelligence},
895 author = {Jobin, Anna and Ienca, Marcello and Vayena, Effy},
896 title = {The global landscape of {AI} ethics guidelines},
897}
898
899@inproceedings{pedreschi2019meaningful,
900 year = {2019},
901 pages = {9780--9784},
902 volume = {33},
903 booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
904 author = {Pedreschi, Dino and Giannotti, Fosca and Guidotti, Riccardo and Monreale, Anna and Ruggieri, Salvatore and Turini, Franco},
905 title = {Meaningful explanations of Black Box {AI} decision systems},
906}
907
908@article{guidotti2018survey,
909 publisher = {ACM New York, NY, USA},
910 year = {2018},
911 pages = {1--42},
912 number = {5},
913 volume = {51},
914 journal = {ACM computing surveys (CSUR)},
915 author = {Guidotti, Riccardo and Monreale, Anna and Ruggieri, Salvatore and Turini, Franco and Giannotti, Fosca and Pedreschi, Dino},
916 title = {A survey of methods for explaining black box models},
917}
918
919@book{IOS2014SQuaRE,
920 publisher = {ISO/IEC},
921 year = {2014},
922 author = {International Organization for Standardization},
923 title = {Systems and Software Engineering: Systems and Software Quality Requirements and Evaluation ({SQuaRE}): Guide to {SQuaRE}},
924}
925
926@article{HLEG2019ethics,
927 note = {Document can be accessed at: https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai},
928 publisher = { European Commission },
929 year = {8 April 2019},
930 author = {High-Level Expert Group on Artificial Intelligence},
931 title = {Ethics guidelines for trustworthy {AI}},
932}
933
934@inproceedings{brun2018software,
935 year = {2018},
936 pages = {754--759},
937 booktitle = {Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
938 author = {Brun, Yuriy and Meliou, Alexandra},
939 title = {Software fairness},
940}
941
942@inproceedings{galhotra2017fairness,
943 year = {2017},
944 pages = {498--510},
945 booktitle = {Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering},
946 author = {Galhotra, Sainyam and Brun, Yuriy and Meliou, Alexandra},
947 title = {Fairness testing: testing software for discrimination},
948}
949
950@article{garvie2016facial,
951 year = {2016},
952 volume = {7},
953 journal = {The Atlantic},
954 author = {Garvie, Clare and Frankle, Jonathan},
955 title = {Facial-recognition software might have a racial bias problem},
956}
957
958@article{mendoza2013role,
959 year = {2013},
960 author = {Mendoza, Antonette and Miller, Tim and Pedell, Sonja and Sterling and others},
961 title = {The role of users’ emotions and associated quality goals on appropriation of systems: two case studies},
962}
963
964@inproceedings{schroeder2015design,
965 organization = {IEEE},
966 year = {2015},
967 pages = {189--198},
968 volume = {2},
969 booktitle = {2015 IEEE/ACM 37th IEEE International Conference on Software Engineering},
970 author = {Schroeder, Jan and Holzner, Daniela and Berger, Christian and Hoel, Carl-Johan and Laine, Leo and Magnusson, Anders},
971 title = {Design and evaluation of a customizable multi-domain reference architecture on top of product lines of self-driving heavy vehicles-an industrial case study},
972}
973
974@inproceedings{mekni2020smart,
975 year = {2020},
976 pages = {1--6},
977 booktitle = {Proceedings of the 3rd International Conference on Applications of Intelligent Systems},
978 author = {Mekni, Mehdi and Baani, Zakaria and Sulieman, Dalia},
979 title = {A Smart Virtual Assistant for Students},
980}
981
982@article{lee2019confident,
983 publisher = {ACM New York, NY, USA},
984 year = {2019},
985 pages = {1--39},
986 number = {1},
987 volume = {27},
988 journal = {ACM Transactions on Computer-Human Interaction (TOCHI)},
989 author = {Lee, Hosub and Kobsa, Alfred},
990 title = {Confident Privacy Decision-Making in {IoT} Environments},
991}
992
993@inproceedings{theosaksomo2019conversational,
994 organization = {IEEE},
995 year = {2019},
996 pages = {154--159},
997 booktitle = {2019 IEEE 13th International Conference on Telecommunication Systems, Services, and Applications (TSSA)},
998 author = {Theosaksomo, David and Widyantoro, Dwi H},
999 title = {Conversational Recommender System Chatbot Based on Functional Requirement},
1000}
1001
1002@inproceedings{agarwal2014expert,
1003 organization = {IEEE},
1004 year = {2014},
1005 pages = {1--4},
1006 booktitle = {International Conference on Recent Advances and Innovations in Engineering (ICRAIE-2014)},
1007 author = {Agarwal, Mahak and Goel, Shivani},
1008 title = {Expert system and it's requirement engineering process},
1009}
1010
1011@inproceedings{singh2017reconcile,
1012 organization = {IEEE},
1013 year = {2017},
1014 pages = {1646--1651},
1015 booktitle = {2017 17th International Conference on Control, Automation and Systems (ICCAS)},
1016 author = {Singh, Madhusudan and Kim, Shiho},
1017 title = {Reconcile security requirements for intelligent vehicles},
1018}
1019
1020@inproceedings{cysneiros2018software,
1021 organization = {IEEE},
1022 year = {2018},
1023 pages = {382--387},
1024 booktitle = {2018 IEEE 26th International Requirements Engineering Conference (RE)},
1025 author = {Cysneiros, Luiz Marcio and Raffi, Majid and do Prado Leite, Julio Cesar Sampaio},
1026 title = {Software transparency as a key requirement for self-driving cars},
1027}
1028
1029@inproceedings{schrader2019approach,
1030 organization = {IEEE},
1031 year = {2019},
1032 pages = {1597--1603},
1033 booktitle = {2019 IEEE Intelligent Vehicles Symposium (IV)},
1034 author = {Schr{\"a}der, Tobias and Stolte, Torben and Jatzkowski, Inga and Graubohm, Robert and Maurer, Markus},
1035 title = {An Approach for a Requirement Analysis for an Autonomous Family Vehicle},
1036}
1037
1038@inproceedings{martin2019graded,
1039 organization = {IEEE},
1040 year = {2019},
1041 pages = {1--6},
1042 booktitle = {2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)},
1043 author = {Martin, Trevor P and Anggraini, Ratih NE},
1044 title = {A Graded Approach to Requirement Satisfaction for Evolving Systems},
1045}
1046
1047@inproceedings{bencomo2019ram,
1048 organization = {IEEE},
1049 year = {2019},
1050 pages = {216--226},
1051 booktitle = {2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS)},
1052 author = {Bencomo, Nelly and Paucar, Luis H Garcia},
1053 title = {{RaM}: Causally-connected and Requirements-aware Runtime Models using Bayesian Learning},
1054}
1055
1056@inproceedings{chazette2019mitigating,
1057 organization = {IEEE},
1058 year = {2019},
1059 pages = {470--475},
1060 booktitle = {2019 IEEE 27th International Requirements Engineering Conference (RE)},
1061 author = {Chazette, Larissa},
1062 title = {Mitigating Challenges in the Elicitation and Analysis of Transparency Requirements},
1063}
1064
1065@inproceedings{fagbola2019towards,
1066 organization = {IEEE},
1067 year = {2019},
1068 pages = {200--204},
1069 booktitle = {2019 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS)},
1070 author = {Fagbola, Temitayo M and Thakur, Surendra C},
1071 title = {Towards the Development of Artificial Intelligence-based Systems: Human-Centered Functional Requirements and Open Problems},
1072}
1073
1074@inproceedings{brugali2019non,
1075 organization = {IEEE},
1076 year = {2019},
1077 pages = {743--748},
1078 booktitle = {2019 IEEE International Conference on Real-time Computing and Robotics (RCAR)},
1079 author = {Brugali, Davide},
1080 title = {Non-Functional Requirements in Robotic Systems: Challenges and State of the Art},
1081}
1082
1083@article{lutje2020requirements,
1084 publisher = {Multidisciplinary Digital Publishing Institute},
1085 year = {2020},
1086 pages = {10},
1087 number = {1},
1088 volume = {10},
1089 journal = {Administrative Sciences},
1090 author = {L{\"u}tje, Anna and Wohlgemuth, Volker},
1091 title = {Requirements Engineering for an Industrial Symbiosis Tool for Industrial Parks Covering System Analysis, Transformation Simulation and Goal Setting},
1092}
1093
1094@inproceedings{woodman2019human,
1095 organization = {IEEE},
1096 year = {2019},
1097 pages = {2371--2376},
1098 booktitle = {2019 IEEE Intelligent Vehicles Symposium (IV)},
1099 author = {Woodman, Roger and Lu, Ke and Higgins, Matthew D and Brewerton, Simon and Jennings, Paul and Birrell, Stewart},
1100 title = {A human factors approach to defining requirements for low-speed autonomous vehicles to enable intelligent platooning},
1101}
1102
1103@incollection{cysneiros2020non,
1104 publisher = {Springer},
1105 year = {2020},
1106 pages = {335--342},
1107 booktitle = {Enterprise, Business-Process and Information Systems Modeling},
1108 author = {Cysneiros, Luiz Marcio and do Prado Leite, Julio Cesar Sampaio},
1109 title = {Non-Functional Requirements Orienting the Development of Socially Responsible Software},
1110}
1111
1112@inproceedings{guizzardi2020ethical,
1113 organization = {Springer},
1114 year = {2020},
1115 pages = {251--256},
1116 booktitle = {Canadian Conference on Artificial Intelligence},
1117 author = {Guizzardi, Renata and Amaral, Glenda and Guizzardi, Giancarlo and Mylopoulos, John},
1118 title = {Ethical Requirements for {AI} Systems},
1119}
1120
1121@article{meinert2020agile,
1122 publisher = {JMIR Publications Inc., Toronto, Canada},
1123 year = {2020},
1124 pages = {e19297},
1125 number = {2},
1126 volume = {6},
1127 journal = {JMIR Public Health and Surveillance},
1128 author = {Meinert, Edward and Milne-Ives, Madison and Surodina, Svitlana and Lam, Ching},
1129 title = {Agile requirements engineering and software planning for a digital health platform to engage the effects of isolation caused by social distancing: case study},
1130}
1131
1132@inproceedings{kaindl2020towards,
1133 year = {2020},
1134 booktitle = {REFSQ Workshops},
1135 author = {Kaindl, Hermann and Ferdigg, Jonas},
1136 title = {Towards Requirements Engineering for Superintelligence Safety},
1137}
1138
1139@inproceedings{kaindl2020towards,
1140 organization = {IEEE},
1141 year = {2020},
1142 pages = {33--38},
1143 booktitle = {2020 IEEE Seventh International Workshop on Artificial Intelligence for Requirements Engineering (AIRE)},
1144 author = {Kaindl, Hermann and Ferdigg, Jonas},
1145 title = {Towards an Extended Requirements Problem Formulation for Superintelligence Safety},
1146}
1147
1148@inproceedings{vergne2020trustworthy,
1149 year = {2020},
1150 booktitle = {REFSQ Workshops},
1151 author = {Vergne, Matthieu},
1152 title = {Trustworthy {AI}: Towards the Golden Age of {RE}?},
1153}
1154
1155@inproceedings{kostova2020interplay,
1156 year = {2020},
1157 booktitle = {REFSQ Workshops},
1158 author = {Kostova, Blagovesta and Gurses, Seda and Wegmann, Alain},
1159 title = {On the Interplay between Requirements, Engineering, and Artificial Intelligence.},
1160}
1161
1162@inproceedings{schafer2013technical,
1163 organization = {IEEE},
1164 year = {2013},
1165 pages = {1--9},
1166 booktitle = {2013 IEEE 18th Conference on Emerging Technologies \& Factory Automation (ETFA)},
1167 author = {Sch{\"a}fer, Stephan and Sch{\"o}ttke, Dirk and K{\"a}mpfe, Thomas and Berger, Ulrich},
1168 title = {Technical conditions for the use of autonomous systems: A general approach on an example},
1169}
1170
1171@article{crnkovic2012robots,
1172 publisher = {Springer},
1173 year = {2012},
1174 pages = {61--71},
1175 number = {1},
1176 volume = {14},
1177 journal = {Ethics and Information Technology},
1178 author = {Crnkovic, Gordana Dodig and {\c{C}}{\"u}r{\"u}kl{\"u}, Baran},
1179 title = {Robots: ethical by design},
1180}
1181
1182@article{kuwajima2020engineering,
1183 publisher = {Springer},
1184 year = {2020},
1185 pages = {1--24},
1186 journal = {Machine Learning},
1187 author = {Kuwajima, Hiroshi and Yasuoka, Hirotoshi and Nakae, Toshihiro},
1188 title = {Engineering problems in machine learning systems},
1189}
1190
1191@inproceedings{arruda2017towards,
1192 organization = {IEEE},
1193 year = {2017},
1194 pages = {2314--2319},
1195 booktitle = {2017 IEEE international conference on big data (big data)},
1196 author = {Arruda, Darlan and Madhavji, Nazim H},
1197 title = {Towards a requirements engineering artefact model in the context of big data software development projects: Research in progress},
1198}
1199
1200@article{felzmann2019transparency,
1201 publisher = {SAGE Publications Sage UK: London, England},
1202 year = {2019},
1203 pages = {2053951719860542},
1204 number = {1},
1205 volume = {6},
1206 journal = {Big Data \& Society},
1207 author = {Felzmann, Heike and Villaronga, Eduard Fosch and Lutz, Christoph and Tam{\`o}-Larrieux, Aurelia},
1208 title = {Transparency you can trust: Transparency requirements for artificial intelligence between legal norms and contextual concerns},
1209}
1210
1211@article{cuer2018formal,
1212 publisher = {Elsevier},
1213 year = {2018},
1214 pages = {29--40},
1215 volume = {174},
1216 journal = {Reliability Engineering \& System Safety},
1217 author = {Cuer, Romain and Pi{\'e}trac, Laurent and Niel, Eric and Diallo, Saidou and Minoiu-Enache, Nicoleta and Dang-Van-Nhan, Christophe},
1218 title = {A formal framework for the safe design of the Autonomous Driving supervision},
1219}
1220
1221@article{chazette2020explainability,
1222 publisher = {Springer},
1223 year = {2020},
1224 pages = {1--22},
1225 journal = {Requirements Engineering},
1226 author = {Chazette, Larissa and Schneider, Kurt},
1227 title = {Explainability as a non-functional requirement: challenges and recommendations},
1228}
1229
1230@book{murphy2012machine,
1231 publisher = {MIT press},
1232 year = {2012},
1233 author = {Murphy, Kevin P},
1234 title = {Machine learning: a probabilistic perspective},
1235}
1236
1237@inproceedings{wheatcraft2018communicating,
1238 organization = {Wiley Online Library},
1239 year = {2018},
1240 pages = {716--732},
1241 number = {1},
1242 volume = {28},
1243 booktitle = {INCOSE International Symposium},
1244 author = {Wheatcraft, Louis S and Ryan, Michael J},
1245 title = {Communicating Requirements--Effectively!},
1246}
1247
1248@article{amershi2014power,
1249 year = {2014},
1250 pages = {105--120},
1251 number = {4},
1252 volume = {35},
1253 journal = {Ai Magazine},
1254 author = {Amershi, Saleema and Cakmak, Maya and Knox, William Bradley and Kulesza, Todd},
1255 title = {Power to the people: The role of humans in interactive machine learning},
1256}
1257
1258@article{how2020artificial,
1259 publisher = {Multidisciplinary Digital Publishing Institute},
1260 year = {2020},
1261 pages = {39},
1262 number = {1},
1263 volume = {11},
1264 journal = {Information},
1265 author = {How, Meng-Leong and Cheah, Sin-Mei and Chan, Yong-Jiet and Khor, Aik Cheow and Say, Eunice Mei Ping},
1266 title = {Artificial intelligence-enhanced decision support for informing global sustainable development: A human-centric {AI}-thinking approach},
1267}
1268
1269@inproceedings{wang2019designing,
1270 year = {2019},
1271 pages = {1--15},
1272 booktitle = {Proceedings of the 2019 CHI conference on human factors in computing systems},
1273 author = {Wang, Danding and Yang, Qian and Abdul, Ashraf and Lim, Brian Y},
1274 title = {Designing theory-driven user-centric explainable {AI}},
1275}
1276
1277@inproceedings{dodge2019explaining,
1278 year = {2019},
1279 pages = {275--285},
1280 booktitle = {Proceedings of the 24th International Conference on Intelligent User Interfaces},
1281 author = {Dodge, Jonathan and Liao, Q Vera and Zhang, Yunfeng and Bellamy, Rachel KE and Dugan, Casey},
1282 title = {Explaining models: an empirical study of how explanations impact fairness judgment},
1283}
1284
1285@inproceedings{alatawi2018psychologically,
1286 organization = {IEEE},
1287 year = {2018},
1288 pages = {41--50},
1289 booktitle = {2018 25th Australasian Software Engineering Conference (ASWEC)},
1290 author = {Alatawi, Eman and Mendoza, Antonette and Miller, Tim},
1291 title = {Psychologically-driven requirements engineering: A case study in depression care},
1292}
1293
1294@inproceedings{teruel2011comparing,
1295 organization = {Springer},
1296 year = {2011},
1297 pages = {169--184},
1298 booktitle = {International Conference on Evaluation of Novel Approaches to Software Engineering},
1299 author = {Teruel, Miguel A and Navarro, Elena and L{\'o}pez-Jaquero, V{\'\i}ctor and Montero, Francisco and Gonz{\'a}lez, Pascual},
1300 title = {Comparing goal-oriented approaches to model requirements for {CSCW}},
1301}
1302
1303@article{dalpiaz2016istar,
1304 year = {2016},
1305 journal = {arXiv preprint arXiv:1605.07767},
1306 author = {Dalpiaz, Fabiano and Franch, Xavier and Horkoff, Jennifer},
1307 title = {istar 2.0 language guide},
1308}
1309
1310@inproceedings{van2001goal,
1311 organization = {IEEE},
1312 year = {2001},
1313 pages = {249--262},
1314 booktitle = {Proceedings fifth ieee international symposium on requirements engineering},
1315 author = {Van Lamsweerde, Axel},
1316 title = {Goal-oriented requirements engineering: A guided tour},
1317}
1318
1319@article{lapouchnian2005goal,
1320 year = {2005},
1321 volume = {32},
1322 journal = {University of Toronto},
1323 author = {Lapouchnian, Alexei},
1324 title = {Goal-oriented requirements engineering: An overview of the current research},
1325}
1326
1327@article{horkoff2013comparison,
1328 publisher = {Springer},
1329 year = {2013},
1330 pages = {199--222},
1331 number = {3},
1332 volume = {18},
1333 journal = {Requirements Engineering},
1334 author = {Horkoff, Jennifer and Yu, Eric},
1335 title = {Comparison and evaluation of goal-oriented satisfaction analysis techniques},
1336}
1337
1338@inproceedings{colomo2010study,
1339 organization = {Springer},
1340 year = {2010},
1341 pages = {1--7},
1342 booktitle = {World Summit on Knowledge Society},
1343 author = {Colomo-Palacios, Ricardo and Hern{\'a}ndez-L{\'o}pez, Adri{\'a}n and Garc{\'\i}a-Crespo, {\'A}ngel and Soto-Acosta, Pedro},
1344 title = {A study of emotions in requirements engineering},
1345}
1346
1347@book{crotty1998foundations,
1348 publisher = {Sage},
1349 year = {1998},
1350 author = {Crotty, Michael},
1351 title = {The foundations of social research: Meaning and perspective in the research process},
1352}
1353
1354@book{holstein2013handbook,
1355 publisher = {Guilford Publications},
1356 year = {2013},
1357 author = {Holstein, James A and Gubrium, Jaber F},
1358 title = {Handbook of constructionist research},
1359}
1360
1361@article{berland2014educational,
1362 publisher = {Springer},
1363 year = {2014},
1364 pages = {205--220},
1365 number = {1-2},
1366 volume = {19},
1367 journal = {Technology, Knowledge and Learning},
1368 author = {Berland, Matthew and Baker, Ryan S and Blikstein, Paulo},
1369 title = {Educational data mining and learning analytics: Applications to constructionist research},
1370}
1371
1372@article{gonccalves2019understanding,
1373 publisher = {Springer},
1374 year = {2019},
1375 pages = {55--84},
1376 number = {1},
1377 volume = {24},
1378 journal = {Requirements Engineering},
1379 author = {Gon{\c{c}}alves, Enyo and de Oliveira, Marcos Ant{\^o}nio and Monteiro, Ingrid and Castro, Jaelson and Ara{\'u}jo, Jo{\~a}o},
1380 title = {Understanding what is important in i{S}tar extension proposals: the viewpoint of researchers},
1381}
1382
1383@book{friedenthal2014practical,
1384 publisher = {Morgan Kaufmann},
1385 year = {2014},
1386 author = {Friedenthal, Sanford and Moore, Alan and Steiner, Rick},
1387 title = {A practical guide to {SysML}: the systems modeling language},
1388}
1389
1390@inproceedings{palomares2018personal,
1391 organization = {Springer},
1392 year = {2018},
1393 pages = {297--304},
1394 booktitle = {International working conference on requirements engineering: foundation for software quality},
1395 author = {Palomares, Cristina and Franch, Xavier and Fucci, Davide},
1396 title = {Personal recommendations in requirements engineering: the {OpenReq} approach},
1397}
1398
1399@book{daugherty2018human,
1400 publisher = {Harvard Business Press},
1401 year = {2018},
1402 author = {Daugherty, Paul R and Wilson, H James},
1403 title = {Human+ machine: Reimagining work in the age of {AI}},
1404}
1405
1406@inproceedings{bach2017data,
1407 organization = {IEEE},
1408 year = {2017},
1409 pages = {1--6},
1410 booktitle = {2017 IEEE International Systems Engineering Symposium (ISSE)},
1411 author = {Bach, Johannes and Langner, Jacob and Otten, Stefan and Holz{\"a}pfel, Marc and Sax, Eric},
1412 title = {Data-driven development, a complementing approach for automotive systems engineering},
1413}
1414
1415@incollection{bennaceur2019requirements,
1416 publisher = {Springer},
1417 year = {2019},
1418 pages = {51--92},
1419 booktitle = {Handbook of Software Engineering},
1420 author = {Bennaceur, Amel and Tun, Thein Than and Yu, Yijun and Nuseibeh, Bashar},
1421 title = {Requirements Engineering},
1422}
1423
1424@book{cooper2004inmates,
1425 publisher = {Sams Indianapolis},
1426 year = {2004},
1427 volume = {2},
1428 author = {Cooper, Alan and others},
1429 title = {The inmates are running the asylum: Why high-tech products drive us crazy and how to restore the sanity},
1430}
1431
1432@inproceedings{kaiya2002agora,
1433 organization = {Ieee},
1434 year = {2002},
1435 pages = {13--22},
1436 booktitle = {Proceedings IEEE joint international conference on requirements engineering},
1437 author = {Kaiya, Haruhiko and Horai, Hisayuki and Saeki, Motoshi},
1438 title = {{AGORA}: Attributed goal-oriented requirements analysis method},
1439}
1440
1441@inproceedings{anton1998GBRAM,
1442 organization = {IEEE},
1443 year = {1998},
1444 pages = {157--166},
1445 booktitle = {Proceedings of the 20th international Conference on Software Engineering},
1446 author = {Ant{\'o}n, Annie I and Potts, Colin},
1447 title = {The use of goals to surface requirements for evolving systems},
1448}
1449
1450@book{chung2012NRL,
1451 publisher = {Springer Science \& Business Media},
1452 year = {2012},
1453 volume = {5},
1454 author = {Chung, Lawrence and Nixon, Brian A and Yu, Eric and Mylopoulos, John},
1455 title = {Non-functional requirements in software engineering},
1456}
1457
1458@article{bresciani2004tropos,
1459 publisher = {Springer},
1460 year = {2004},
1461 pages = {203--236},
1462 number = {3},
1463 volume = {8},
1464 journal = {Autonomous Agents and Multi-Agent Systems},
1465 author = {Bresciani, Paolo and Perini, Anna and Giorgini, Paolo and Giunchiglia, Fausto and Mylopoulos, John},
1466 title = {Tropos: An agent-oriented software development methodology},
1467}
1468
1469@article{liu2004designing,
1470 publisher = {Elsevier},
1471 year = {2004},
1472 pages = {187--203},
1473 number = {2},
1474 volume = {29},
1475 journal = {Information systems},
1476 author = {Liu, Lin and Yu, Eric},
1477 title = {Designing information systems in social context: a goal and scenario modelling approach},
1478}
1479
1480@article{agrawal2019cloudy,
1481 year = {2019},
1482 journal = {10th Annual Conference on Innovative Data Systems Research (CIDR ‘20)},
1483 author = {Agrawal, Ashvin and Chatterjee, Rony and Curino, Carlo and Floratou, Avrilia and Gowdal, Neha and Interlandi, Matteo and Jindal, Alekh and Karanasos, Kostantinos and Krishnan, Subru and Kroth, Brian and others},
1484 title = {Cloudy with high chance of {DBMS}: A 10-year prediction for Enterprise-Grade {ML}},
1485}
1486
1487@article{amyot2011grl,
1488 year = {2011},
1489 pages = {160--162},
1490 volume = {766},
1491 journal = {iStar},
1492 author = {Amyot, Daniel and Mussbacher, Gunter and Ghanavati, Sepideh and Kealey, Jason},
1493 title = {{GRL} Modeling and Analysis with {jUCMNav}},
1494}
1495
1496@inproceedings{baresi2010fuzzy,
1497 organization = {IEEE},
1498 year = {2010},
1499 pages = {125--134},
1500 booktitle = {2010 18th IEEE international requirements engineering conference},
1501 author = {Baresi, Luciano and Pasquale, Liliana and Spoletini, Paola},
1502 title = {Fuzzy goals for requirements-driven adaptation},
1503}
1504
1505@incollection{bartocci2018STL,
1506 publisher = {Springer},
1507 year = {2018},
1508 pages = {135--175},
1509 booktitle = {Lectures on Runtime Verification},
1510 author = {Bartocci, Ezio and Deshmukh, Jyotirmoy and Donz{\'e}, Alexandre and Fainekos, Georgios and Maler, Oded and Ni{\v{c}}kovi{\'c}, Dejan and Sankaranarayanan, Sriram},
1511 title = {Specification-based monitoring of cyber-physical systems: a survey on theory, tools and applications},
1512}
1513
1514@article{bellamy2019Fairness360,
1515 publisher = {IBM},
1516 year = {2019},
1517 pages = {4--1},
1518 number = {4/5},
1519 volume = {63},
1520 journal = {IBM Journal of Research and Development},
1521 author = {Bellamy, Rachel KE and Dey, Kuntal and Hind, Michael and Hoffman, Samuel C and Houde, Stephanie and Kannan, Kalapriya and Lohia, Pranay and Martino, Jacquelyn and Mehta, Sameep and Mojsilovi{\'c}, Aleksandra and others},
1522 title = {A{I} Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias},
1523}
1524
1525@inproceedings{tramer2017fairtest,
1526 organization = {IEEE},
1527 year = {2017},
1528 pages = {401--416},
1529 booktitle = {2017 IEEE European Symposium on Security and Privacy (EuroS\&P)},
1530 author = {Tramer, Florian and Atlidakis, Vaggelis and Geambasu, Roxana and Hsu, Daniel and Hubaux, Jean-Pierre and Humbert, Mathias and Juels, Ari and Lin, Huang},
1531 title = {Fairtest: Discovering unwarranted associations in data-driven applications},
1532}
1533
1534@book{urquhart2012grounded,
1535 publisher = {Sage},
1536 year = {2012},
1537 author = {Urquhart, Cathy},
1538 title = {Grounded theory for qualitative research: A practical guide},
1539}
1540
1541@inproceedings{wohlin2014guidelines,
1542 year = {2014},
1543 pages = {1--10},
1544 booktitle = {Proceedings of the 18th international conference on evaluation and assessment in software engineering},
1545 author = {Wohlin, Claes},
1546 title = {Guidelines for snowballing in systematic literature studies and a replication in software engineering},
1547}
1548
1549@incollection{easterbrook2008selecting,
1550 publisher = {Springer},
1551 year = {2008},
1552 pages = {285--311},
1553 booktitle = {Guide to advanced empirical software engineering},
1554 author = {Easterbrook, Steve and Singer, Janice and Storey, Margaret-Anne and Damian, Daniela},
1555 title = {Selecting empirical methods for software engineering research},
1556}
1557
1558@inproceedings{vassev2013autonomy,
1559 organization = {IEEE},
1560 year = {2013},
1561 pages = {1--10},
1562 booktitle = {16th IEEE International Symposium on Object/component/service-oriented Real-time distributed Computing (ISORC 2013)},
1563 author = {Vassev, Emil and Hinchey, Mike},
1564 title = {On the autonomy requirements for space missions},
1565}
1566
1567@inproceedings{nuseibeh2000requirements,
1568 year = {2000},
1569 pages = {35--46},
1570 booktitle = {Proceedings of the Conference on the Future of Software Engineering},
1571 author = {Nuseibeh, Bashar and Easterbrook, Steve},
1572 title = {Requirements engineering: a roadmap},
1573}
1574
1575@article{tellis1997application,
1576 year = {1997},
1577 pages = {1--19},
1578 number = {3},
1579 volume = {3},
1580 journal = {The qualitative report},
1581 author = {Tellis, Winston},
1582 title = {Application of a case study methodology},
1583}
1584
1585@article{meyer2001case,
1586 publisher = {Sage Publications Sage CA: Thousand Oaks, CA},
1587 year = {2001},
1588 pages = {329--352},
1589 number = {4},
1590 volume = {13},
1591 journal = {Field methods},
1592 author = {Meyer, Christine Benedichte},
1593 title = {A case in case study methodology},
1594}
1595
1596@article{glasow2005fundamentals,
1597 year = {2005},
1598 pages = {2013},
1599 volume = {18},
1600 journal = {Retrieved January},
1601 author = {Glasow, Priscilla A},
1602 title = {Fundamentals of survey research methodology},
1603}
1604
1605@article{gable1994integrating,
1606 publisher = {Taylor \& Francis},
1607 year = {1994},
1608 pages = {112--126},
1609 number = {2},
1610 volume = {3},
1611 journal = {European journal of information systems},
1612 author = {Gable, Guy G},
1613 title = {Integrating case study and survey research methods: an example in information systems},
1614}
1615
1616@inproceedings{yu2020bdd100k,
1617 year = {2020},
1618 pages = {2636--2645},
1619 booktitle = {Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
1620 author = {Yu, Fisher and Chen, Haofeng and Wang, Xin and Xian, Wenqi and Chen, Yingying and Liu, Fangchen and Madhavan, Vashisht and Darrell, Trevor},
1621 title = {Bdd100k: A diverse driving dataset for heterogeneous multitask learning},
1622}
1623
1624@inproceedings{viyovic2014sirius,
1625 organization = {IEEE},
1626 year = {2014},
1627 pages = {233--238},
1628 booktitle = {IEEE 18th International Conference on Intelligent Engineering Systems INES 2014},
1629 author = {Viyovi{\'c}, Vladimir and Maksimovi{\'c}, Mirjam and Perisi{\'c}, Branko},
1630 title = {Sirius: A rapid development of DSM graphical editor},
1631}
1632
1633@inproceedings{bosch2018takes,
1634 year = {2018},
1635 pages = {177--192},
1636 booktitle = {SiBW},
1637 author = {Bosch, Jan and Olsson, Helena Holmstr{\"o}m and Crnkovic, Ivica},
1638 title = {It takes three to tango: Requirement, outcome/data, and {AI} driven development.},
1639}
1640
1641@inproceedings{bonfe2012towards,
1642 organization = {IEEE},
1643 year = {2012},
1644 pages = {56--61},
1645 booktitle = {2012 4th IEEE RAS \& EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob)},
1646 author = {Bonfe, Marcello and Boriero, Fabrizio and Dodi, Riccardo and Fiorini, Paolo and Morandi, Angelica and Muradore, Riccardo and Pasquale, Liliana and Sanna, Alberto and Secchi, Cristian},
1647 title = {Towards automated surgical robotics: A requirements engineering approach},
1648}
1649
1650@inproceedings{altarturi2017requirement,
1651 organization = {IEEE},
1652 year = {2017},
1653 pages = {111--117},
1654 booktitle = {2017 IEEE Conference on Big Data and Analytics (ICBDA)},
1655 author = {Altarturi, Hamza Hussein and Ng, Keng-Yap and Ninggal, Mohd Izuan Hafez and Nazri, Azree Shahrel Ahmad and Abd Ghani, Abdul Azim},
1656 title = {A requirement engineering model for big data software},
1657}
1658
1659@inproceedings{hu2020towards,
1660 organization = {IEEE},
1661 year = {2020},
1662 pages = {},
1663 booktitle = {Workshop on Artificial Intelligence for Requirements Engineering (AIRE)},
1664 author = {Hu, Boyue Caroline and Salay, Rick and Czarnecki, Krzysztof and Rahimi, Mona and Selim, Gehan and Chechik, Marsha},
1665 title = {Towards Requirements Specification for Machine-learned Perception Based on Human Performance},
1666}
1667
1668@inproceedings{gruber2017integrated,
1669 organization = {IEEE},
1670 year = {2017},
1671 pages = {27--31},
1672 booktitle = {2017 7th IEEE International Conference on System Engineering and Technology (ICSET)},
1673 author = {Gruber, Kristina and Huemer, Jakob and Zimmermann, Armin and Maschotta, Ralph},
1674 title = {Integrated description of functional and non-functional requirements for automotive systems design using {SysML}},
1675}
1676
1677@inproceedings{sandkuhl2019putting,
1678 organization = {IEEE},
1679 year = {2019},
1680 pages = {157--164},
1681 volume = {1},
1682 booktitle = {2019 IEEE 21st Conference on Business Informatics (CBI)},
1683 author = {Sandkuhl, Kurt},
1684 title = {Putting {AI} into Context-Method Support for the Introduction of Artificial Intelligence into Organizations},
1685}
1686
1687@inproceedings{rahimi2019toward,
1688 organization = {IEEE},
1689 year = {2019},
1690 pages = {241--244},
1691 booktitle = {2019 IEEE 27th International Requirements Engineering Conference Workshops (REW)},
1692 author = {Rahimi, Mona and Guo, Jin LC and Kokaly, Sahar and Chechik, Marsha},
1693 title = {Toward Requirements Specification for Machine-Learned Components},
1694}
1695
1696@inproceedings{jakob2017defining,
1697 organization = {IEEE},
1698 year = {2017},
1699 pages = {128--133},
1700 booktitle = {2017 IEEE 14th International Scientific Conference on Informatics},
1701 author = {Jakob, Judith and Kugele, Kordula and Tick, J{\'o}zsef},
1702 title = {Defining camera-based traffic scenarios and use cases for the visually impaired by means of expert interviews},
1703}
1704
1705@article{vogelsang2019requirements,
1706 year = {2019},
1707 journal = {IEEE International Requirements Engineering Conference Workshops},
1708 author = {Vogelsang, Andreas and Borg, Markus},
1709 title = {Requirements Engineering for Machine Learning: Perspectives from Data Scientists},
1710}
1711
1712@article{tuncali2019requirements,
1713 publisher = {IEEE},
1714 year = {2019},
1715 pages = {265--280},
1716 number = {2},
1717 volume = {5},
1718 journal = {IEEE Transactions on Intelligent Vehicles},
1719 author = {Tuncali, Cumhur Erkan and Fainekos, Georgios and Prokhorov, Danil and Ito, Hisahiro and Kapinski, James},
1720 title = {Requirements-driven test generation for autonomous vehicles with machine learning components},
1721}
1722
1723@inproceedings{becker2020partial,
1724 year = {2020},
1725 booktitle = {Software Engineering (Workshops)},
1726 author = {Becker, Jan Steffen},
1727 title = {Partial Consistency for Requirement Engineering with Traffic Sequence Charts.},
1728}
1729
1730@article{shin2019data,
1731 publisher = {Multidisciplinary Digital Publishing Institute},
1732 year = {2019},
1733 pages = {1696},
1734 number = {9},
1735 volume = {12},
1736 journal = {Energies},
1737 author = {Shin, Changho and Rho, Seungeun and Lee, Hyoseop and Rhee, Wonjong},
1738 title = {Data requirements for applying machine learning to energy disaggregation},
1739}
1740
1741@article{dimitrakopoulos2019alpha,
1742 publisher = {Elsevier},
1743 year = {2019},
1744 pages = {28--47},
1745 volume = {91},
1746 journal = {Simulation Modelling Practice and Theory},
1747 author = {Dimitrakopoulos, George and Kavakli, Evangelia and Loucopoulos, Peri and Anagnostopoulos, Dimosthenis and Zographos, Theodoros},
1748 title = {A capability-oriented modelling and simulation approach for autonomous vehicle management},
1749}
1750
1751@article{weihrauch2018conceptual,
1752 publisher = {Elsevier},
1753 year = {2018},
1754 pages = {386--391},
1755 volume = {67},
1756 journal = {Procedia CIRP},
1757 author = {Weihrauch, Dennis and Schindler, Paul Anton and Sihn, Wilfried},
1758 title = {A conceptual model for developing a smart process control system},
1759}
1760
1761@article{fenn2016addressing,
1762 publisher = {Springer},
1763 year = {2016},
1764 pages = {77--86},
1765 number = {1},
1766 volume = {27},
1767 journal = {Machine Vision and Applications},
1768 author = {Fenn, Shannon and Mendes, Alexandre and Budden, David M},
1769 title = {Addressing the non-functional requirements of computer vision systems: a case study},
1770}
1771
1772@article{neace2018goal,
1773 publisher = {Springer},
1774 year = {2018},
1775 pages = {509--555},
1776 number = {4},
1777 volume = {23},
1778 journal = {Requirements Engineering},
1779 author = {Neace, Kerry and Roncace, Robert and Fomin, Pavel},
1780 title = {Goal model analysis of autonomy requirements for Unmanned Aircraft Systems},
1781}
1782
1783@inproceedings{lockerbie2020using,
1784 year = {2020},
1785 booktitle = {REFSQ Workshops},
1786 author = {Lockerbie, James and Maiden, Neil AM},
1787 title = {Using a Requirements Modelling Language to Co-Design Intelligent Support for People Living with Dementia.},
1788}
1789
1790@inproceedings{nakamichi2020requirements,
1791 organization = {IEEE},
1792 year = {2020},
1793 pages = {260--270},
1794 booktitle = {2020 IEEE 28th International Requirements Engineering Conference (RE)},
1795 author = {Nakamichi, Koji and Ohashi, Kyoko and Namba, Isao and Yamamoto, Rieko and Aoyama, Mikio and Joeckel, Lisa and Siebert, Julien and Heidrich, Jens},
1796 title = {Requirements-driven method to determine quality characteristics and measurements for machine learning software and its evaluation},
1797}
1798
1799@inproceedings{challa2020faulty,
1800 organization = {IEEE},
1801 year = {2020},
1802 pages = {61--69},
1803 booktitle = {2020 IEEE Seventh International Workshop on Artificial Intelligence for Requirements Engineering (AIRE)},
1804 author = {Challa, Harshitha and Niu, Nan and Johnson, Reese},
1805 title = {Faulty Requirements Made Valuable: On the Role of Data Quality in Deep Learning},
1806}
1807
1808@inproceedings{aydemir2018roadmap,
1809 organization = {IEEE},
1810 year = {2018},
1811 pages = {15--21},
1812 booktitle = {2018 IEEE/ACM International Workshop on Software Fairness (FairWare)},
1813 author = {Aydemir, Fatma Basak and Dalpiaz, Fabiano},
1814 title = {A roadmap for ethics-aware software engineering},
1815}
1816
1817@inproceedings{horkoff2019nonFunctional,
1818 organization = {IEEE},
1819 year = {2019},
1820 pages = {386--391},
1821 booktitle = {2019 IEEE 27th International Requirements Engineering Conference (RE)},
1822 author = {Horkoff, Jennifer},
1823 title = {Non-functional requirements for machine learning: Challenges and new directions},
1824}
1825
1826@inproceedings{kohl2019explainability,
1827 organization = {IEEE},
1828 year = {2019},
1829 pages = {363--368},
1830 booktitle = {2019 IEEE 27th International Requirements Engineering Conference (RE)},
1831 author = {K{\"o}hl, Maximilian A and Baum, Kevin and Langer, Markus and Oster, Daniel and Speith, Timo and Bohlender, Dimitri},
1832 title = {Explainability as a non-functional requirement},
1833}
1834
1835@inproceedings{ang2011requirement,
1836 organization = {IEEE},
1837 year = {2011},
1838 pages = {640--645},
1839 booktitle = {2011 IEEE Symposium on Computers \& Informatics},
1840 author = {Ang, Jac Ky and Leong, Sook Bing and Lee, Chin Fei and Yusof, Umi Kalsom},
1841 title = {Requirement engineering techniques in developing expert systems},
1842}
1843
1844@inproceedings{ishikawa2020evidence,
1845 organization = {IEEE},
1846 year = {2020},
1847 pages = {346--351},
1848 booktitle = {2020 IEEE 28th International Requirements Engineering Conference (RE)},
1849 author = {Ishikawa, Fuyuki and Matsuno, Yutaka},
1850 title = {Evidence-driven Requirements Engineering for Uncertainty of Machine Learning-based Systems},
1851}
1852
1853@inproceedings{kuwajima2019adapting,
1854 organization = {IEEE},
1855 year = {2019},
1856 pages = {13--18},
1857 booktitle = {2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)},
1858 author = {Kuwajima, Hiroshi and Ishikawa, Fuyuki},
1859 title = {Adapting {SQuaRE} for Quality Assessment of Artificial Intelligence Systems},
1860}
1861
1862@inproceedings{amaral2020ontology,
1863 organization = {Springer},
1864 year = {2020},
1865 pages = {342--352},
1866 booktitle = {International Conference on Conceptual Modeling},
1867 author = {Amaral, Glenda and Guizzardi, Renata and Guizzardi, Giancarlo and Mylopoulos, John},
1868 title = {Ontology-Based Modeling and Analysis of Trustworthiness Requirements: Preliminary Results},
1869}
1870
1871@inproceedings{bruno2013functional,
1872 organization = {IEEE},
1873 year = {2013},
1874 pages = {768--773},
1875 booktitle = {2013 IEEE RO-MAN},
1876 author = {Bruno, Barbara and Mastrogiovanni, Fulvio and Sgorbissa, Antonio},
1877 title = {Functional requirements and design issues for a socially assistive robot for elderly people with mild cognitive impairments},
1878}
1879
1880@inproceedings{ries2021mde,
1881 organization = {SCITEPRESS},
1882 year = {2021},
1883 pages = {41--52},
1884 booktitle = {Proceedings of the 9th International Conference on Model-Driven Engineering and Software Development},
1885 author = {Ries, Benoit and Guelfi, Nicolas and Jahic, Benjamin},
1886 title = {An {MDE} Method for Improving Deep Learning Dataset Requirements Engineering using Alloy and {UML}},
1887}
1888
1889@inproceedings{olmos2020helping,
1890 organization = {IEEE},
1891 year = {2020},
1892 pages = {1--10},
1893 booktitle = {2020 8th International Conference in Software Engineering Research and Innovation (CONISOFT)},
1894 author = {Olmos-S{\'a}nchez, Karla and Rodas-Osollo, Jorge},
1895 title = {Helping organizations manage the innovation process to join the Cognitive era},
1896}
1897
1898@inproceedings{agrawal2020model,
1899 organization = {IEEE},
1900 year = {2020},
1901 pages = {1--10},
1902 booktitle = {2020 IEEE Tenth International Model-Driven Requirements Engineering (MoDRE)},
1903 author = {Agrawal, Ankit and Cleland-Huang, Jane and Stegh{\"o}fer, Jan-Philipp},
1904 title = {Model-driven requirements for humans-on-the-loop multi-uav missions},
1905}
1906
1907@inproceedings{samin2021towards,
1908 year = {2021},
1909 pages = {1328--1337},
1910 booktitle = {Proceedings of the 36th Annual ACM Symposium on Applied Computing},
1911 author = {Samin, Huma and Paucar, Luis H Garcia and Bencomo, Nelly and Sawyer, Peter},
1912 title = {Towards priority-awareness in autonomous intelligent systems},
1913}
1914
1915@inproceedings{cirqueira2020scenario,
1916 organization = {Springer},
1917 year = {2020},
1918 pages = {321--341},
1919 booktitle = {International Cross-Domain Conference for Machine Learning and Knowledge Extraction},
1920 author = {Cirqueira, Douglas and Nedbal, Dietmar and Helfert, Markus and Bezbradica, Marija},
1921 title = {Scenario-based requirements elicitation for user-centric explainable ai},
1922}
1923
1924@article{ntakolia2020user,
1925 publisher = {Springer},
1926 year = {2020},
1927 pages = {1--26},
1928 journal = {Universal Access in the Information Society},
1929 author = {Ntakolia, Charis and Dimas, George and Iakovidis, Dimitris K},
1930 title = {User-centered system design for assisted navigation of visually impaired individuals in outdoor cultural environments},
1931}
1932
1933@article{nalchigar2021modeling,
1934 publisher = {Springer},
1935 year = {2021},
1936 pages = {237--254},
1937 number = {2},
1938 volume = {26},
1939 journal = {Requirements Engineering},
1940 author = {Nalchigar, Soroosh and Yu, Eric and Keshavjee, Karim},
1941 title = {Modeling machine learning requirements from three perspectives: a case report from the healthcare domain},
1942}
1943
1944@article{nalchigar2018business,
1945 publisher = {Elsevier},
1946 year = {2018},
1947 pages = {359--372},
1948 volume = {117},
1949 journal = {Data \& Knowledge Engineering},
1950 author = {Nalchigar, Soroosh and Yu, Eric},
1951 title = {Business-driven data analytics: a conceptual modeling framework},
1952}
1953
1954@article{schoonderwoerd2021human,
1955 publisher = {Elsevier},
1956 year = {2021},
1957 pages = {102684},
1958 journal = {International Journal of Human-Computer Studies},
1959 author = {Schoonderwoerd, Tjeerd AJ and Jorritsma, Wiard and Neerincx, Mark A and van den Bosch, Karel},
1960 title = {Human-Centered XAI: Developing Design Patterns for Explanations of Clinical Decision Support Systems},
1961}
1962
1963@inproceedings{islam2021mobile,
1964 organization = {Springer},
1965 year = {2021},
1966 pages = {33--42},
1967 booktitle = {International Conference on Computational Intelligence in Information System},
1968 author = {Islam, Muhammad Nazrul and Khan, Shahriar Rahman and Islam, Noor Nafiz and Rezwan-A-Rownok, Md and Zaman, Syed Rohit and Zaman, Samiha Raisa},
1969 title = {A Mobile Application for Mental Health Care During COVID-19 Pandemic: Development and Usability Evaluation with System Usability Scale},
1970}
1971
1972@inproceedings{cleland2020requirements,
1973 year = {2020},
1974 pages = {1--12},
1975 booktitle = {Proceedings of the 24th ACM Conference on Systems and Software Product Line: Volume A-Volume A},
1976 author = {Cleland-Huang, Jane and Agrawal, Ankit and Islam, Md Nafee Al and Tsai, Eric and Van Speybroeck, Maxime and Vierhauser, Michael},
1977 title = {Requirements-driven configuration of emergency response missions with small aerial vehicles},
1978}
1979
1980@article{clauer2021usage,
1981 publisher = {Elsevier},
1982 year = {2021},
1983 pages = {242--247},
1984 volume = {96},
1985 journal = {Procedia CIRP},
1986 author = {Clauer, Dana and Fottner, Johannes and Rauch, Erwin and Pr{\"u}glmeier, Marco},
1987 title = {Usage of Autonomous Mobile Robots Outdoors-an Axiomatic Design Approach},
1988}
1989
1990@inproceedings{hall2019systematic,
1991 year = {2019},
1992 volume = {11},
1993 booktitle = {Proceedings of the IJCAI Workshop on eXplainable Artificial Intelligence (XAI 2019), Macau, China},
1994 author = {Hall, Mark and Harborne, Daniel and Tomsett, Richard and Galetic, Vedran and Quintana-Amate, Santiago and Nottle, Alistair and Preece, Alun},
1995 title = {A systematic method to understand requirements for explainable AI (XAI) systems},
1996}
1997
1998@article{habibullah2021non,
1999 year = {2021},
2000 journal = {IEEE 29th International Requirements Engineering Conference (RE)},
2001 author = {Habibullah, Khan Mohammad and Horkoff, Jennifer},
2002 title = {Non-functional Requirements for Machine Learning: Understanding Current Use and Challenges in Industry},
2003}
2004
2005@inproceedings{rivero2021lessons,
2006 organization = {Springer},
2007 year = {2021},
2008 pages = {227--243},
2009 booktitle = {International Conference on Human-Computer Interaction},
2010 author = {Rivero, Luis and Portela, Carlos and Boaro, Jos{\'e} and Santos, Pedro and Rego, Venicius and Braz Junior, Geraldo and Paiva, Anselmo and Alves, Erika and Oliveira, Milton and Moraes, Renato and others},
2011 title = {Lessons Learned from Applying Requirements and Design Techniques in the Development of a Machine Learning System for Predicting Lawsuits Against Power Companies},
2012}
2013
2014@inproceedings{khatamino2021nlp,
2015 year = {2021},
2016 booktitle = {REFSQ Workshops},
2017 author = {Khatamino, Pedram and Camli, Mustafa Baris and {\"O}ztekin, Bilgehan and Gozumoglu, Umutcan and Tortumlu, Emre and Gezer, Huseyin Murat},
2018 title = {An NLP-based Chatbot to Facilitate RE Activities: An Experience Paper on Human Resources Application.},
2019}
2020
2021@inproceedings{schwammberger2021quest,
2022 organization = {IEEE},
2023 year = {2021},
2024 pages = {195--199},
2025 booktitle = {2021 IEEE 29th International Requirements Engineering Conference Workshops (REW)},
2026 author = {Schwammberger, Maike},
2027 title = {A Quest of Self-Explainability: When Causal Diagrams meet Autonomous Urban Traffic Manoeuvres},
2028}
2029
2030@inproceedings{ahmad2021s,
2031 organization = {IEEE},
2032 year = {2021},
2033 pages = {1--12},
2034 booktitle = {2021 IEEE 29th International Requirements Engineering Conference (RE)},
2035 author = {Ahmad, Khlood and Bano, Muneera and Abdelrazek, Mohamed and Arora, Chetan and Grundy, John},
2036 title = {What’s up with Requirements Engineering for Artificial Intelligence Systems?},
2037}
2038
2039@article{shneiderman2021human,
2040 publisher = {Issues in Science and Technology},
2041 year = {2021},
2042 pages = {56--61},
2043 number = {2},
2044 volume = {37},
2045 journal = {Issues in Science and Technology},
2046 author = {Shneiderman, Ben},
2047 title = {Human-centered AI},
2048}
2049
2050@incollection{frank2013domain,
2051 publisher = {Springer},
2052 year = {2013},
2053 pages = {133--157},
2054 booktitle = {Domain engineering},
2055 author = {Frank, Ulrich},
2056 title = {Domain-specific modeling languages: requirements analysis and design guidelines},
2057}
2058
2059@inproceedings{villamizar2021requirements,
2060 organization = {IEEE},
2061 year = {2021},
2062 pages = {29--36},
2063 booktitle = {2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)},
2064 author = {Villamizar, Hugo and Escovedo, Tatiana and Kalinowski, Marcos},
2065 title = {Requirements engineering for machine learning: A systematic mapping study},
2066}
2067
2068@article{sokol2020one,
2069 publisher = {Springer},
2070 year = {2020},
2071 pages = {235--250},
2072 number = {2},
2073 volume = {34},
2074 journal = {KI-K{\"u}nstliche Intelligenz},
2075 author = {Sokol, Kacper and Flach, Peter},
2076 title = {One explanation does not fit all},
2077}
2078
2079@article{felfernig2000uml,
2080 publisher = {World Scientific},
2081 year = {2000},
2082 pages = {449--469},
2083 number = {04},
2084 volume = {10},
2085 journal = {International Journal of Software Engineering and Knowledge Engineering},
2086 author = {Felfernig, Alexander and Friedrich, Gerhard E and Jannach, Dietmar},
2087 title = {UML as domain specific language for the construction of knowledge-based configuration systems},
2088}
2089
2090@inproceedings{silva2018requirements,
2091 year = {2018},
2092 booktitle = {proc. of the XXII Brazilian congress on automation. Brazilian Society of Automation},
2093 author = {Silva, J and Salmon, A and del Foyo, P and Silva, J and Mec{\^a}nica, Depto de Eng},
2094 title = {Requirements engineering at a glance: Comparing gore and uml methods in the design of automated systems},
2095}
2096
2097@book{kelly2008domain,
2098 publisher = {John Wiley \& Sons},
2099 year = {2008},
2100 author = {Kelly, Steven and Tolvanen, Juha-Pekka},
2101 title = {Domain-specific modeling: enabling full code generation},
2102}
2103
2104@book{fowler2004uml,
2105 publisher = {Addison-Wesley Professional},
2106 year = {2004},
2107 author = {Fowler, Martin},
2108 title = {UML distilled: a brief guide to the standard object modeling language},
2109}
2110
2111@book{fowler2010domain,
2112 publisher = {Pearson Education},
2113 year = {2010},
2114 author = {Fowler, Martin},
2115 title = {Domain-specific languages},
2116}
2117
2118@inproceedings{hause2006sysml,
2119 year = {2006},
2120 pages = {1--12},
2121 volume = {9},
2122 booktitle = {Fifteenth European Systems Engineering Conference},
2123 author = {Hause, Matthew and others},
2124 title = {The SysML modelling language},
2125}
2126
2127@inproceedings{berry2022requirements,
2128 organization = {Springer},
2129 year = {2022},
2130 pages = {19--25},
2131 booktitle = {International Working Conference on Requirements Engineering: Foundation for Software Quality},
2132 author = {Berry, Daniel M},
2133 title = {Requirements Engineering for Artificial Intelligence: What Is a Requirements Specification for an Artificial Intelligence?},
2134}
2135
2136@inproceedings{balasubramaniam2022transparency,
2137 organization = {Springer},
2138 year = {2022},
2139 pages = {3--18},
2140 booktitle = {International Working Conference on Requirements Engineering: Foundation for Software Quality},
2141 author = {Balasubramaniam, Nagadivya and Kauppinen, Marjo and Hiekkanen, Kari and Kujala, Sari},
2142 title = {Transparency and Explainability of AI Systems: Ethical Guidelines in Practice},
2143}
2144
2145@article{mihelj2014virtual,
2146 publisher = {Springer},
2147 year = {2014},
2148 author = {Mihelj, Matja{\v{z}} and Novak, Domen and Begu{\v{s}}, Samo},
2149 title = {Virtual reality technology and applications},
2150}
2151
2152@article{miller2020personal,
2153 publisher = {Nature Publishing Group},
2154 year = {2020},
2155 pages = {1--10},
2156 number = {1},
2157 volume = {10},
2158 journal = {Scientific Reports},
2159 author = {Miller, Mark Roman and Herrera, Fernanda and Jun, Hanseul and Landay, James A and Bailenson, Jeremy N},
2160 title = {Personal identifiability of user tracking data during observation of 360-degree VR video},
2161}
2162
2163@inproceedings{tran2017evaluation,
2164 organization = {IEEE},
2165 year = {2017},
2166 pages = {7--11},
2167 booktitle = {2017 ninth international conference on ubiquitous and future networks (ICUFN)},
2168 author = {Tran, Huyen TT and Ngoc, Nam Pham and Bui, Cuong Manh and Pham, Minh Hong and Thang, Truong Cong},
2169 title = {An evaluation of quality metrics for 360 videos},
2170}
2171
2172@article{kawaguchi2019effect,
2173 publisher = {MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info~…},
2174 year = {2019},
2175 pages = {1462--1498},
2176 number = {7},
2177 volume = {31},
2178 journal = {Neural computation},
2179 author = {Kawaguchi, Kenji and Huang, Jiaoyang and Kaelbling, Leslie Pack},
2180 title = {Effect of depth and width on local minima in deep learning},
2181}
2182
2183@inproceedings{safran2017depth,
2184 organization = {PMLR},
2185 year = {2017},
2186 pages = {2979--2987},
2187 booktitle = {International conference on machine learning},
2188 author = {Safran, Itay and Shamir, Ohad},
2189 title = {Depth-width tradeoffs in approximating natural functions with neural networks},
2190}
2191
2192@article{fan2019survey,
2193 publisher = {ACM New York, NY, USA},
2194 year = {2019},
2195 pages = {1--36},
2196 number = {4},
2197 volume = {52},
2198 journal = {ACM Computing Surveys (CSUR)},
2199 author = {Fan, Ching-Ling and Lo, Wen-Chih and Pai, Yu-Tung and Hsu, Cheng-Hsin},
2200 title = {A survey on 360 video streaming: Acquisition, transmission, and display},
2201}
2202
2203@article{anwar2020subjective,
2204 publisher = {IEEE},
2205 year = {2020},
2206 pages = {148084--148099},
2207 volume = {8},
2208 journal = {IEEE Access},
2209 author = {Anwar, Muhammad Shahid and Wang, Jing and Khan, Wahab and Ullah, Asad and Ahmad, Sadique and Fei, Zesong},
2210 title = {Subjective QoE of 360-degree virtual reality videos and machine learning predictions},
2211}
2212
2213@inproceedings{huang20176,
2214 organization = {IEEE},
2215 year = {2017},
2216 pages = {37--44},
2217 booktitle = {2017 IEEE Virtual Reality (VR)},
2218 author = {Huang, Jingwei and Chen, Zhili and Ceylan, Duygu and Jin, Hailin},
2219 title = {6-DOF VR videos with a single 360-camera},
2220}
2221
2222@article{roberto2019visual,
2223 publisher = {IEEE},
2224 year = {2019},
2225 pages = {2524--2537},
2226 number = {8},
2227 volume = {30},
2228 journal = {IEEE Transactions on Circuits and Systems for Video Technology},
2229 author = {Roberto, G de A and Birkbeck, Neil and De Simone, Francesca and Janatra, Ivan and Adsumilli, Balu and Frossard, Pascal},
2230 title = {Visual distortions in 360° videos},
2231}
2232
2233@article{wan2019does,
2234 publisher = {IEEE},
2235 year = {2019},
2236 pages = {1857--1871},
2237 number = {9},
2238 volume = {47},
2239 journal = {IEEE Transactions on Software Engineering},
2240 author = {Wan, Zhiyuan and Xia, Xin and Lo, David and Murphy, Gail C},
2241 title = {How does machine learning change software development practices?},
2242}
2243
2244@article{ahmad2022mapping,
2245 year = {2022},
2246 journal = {arXiv preprint arXiv:2212.10693},
2247 author = {Ahmad, Khlood and Abdelrazek, Mohamed and Arora, Chetan and Bano, Muneera and Grundy, John},
2248 title = {Requirements Engineering for Artificial Intelligence Systems: A Systematic Mapping Study},
2249}
2250
2251@article{ahmad2023requirements,
2252 year = {2023},
2253 journal = {arXiv preprint arXiv:2301.10404 (accepted at: Applied Soft Computing Journal)},
2254 author = {Ahmad, Khlood and Abdelrazek, Mohamed and Arora, Chetan and Bano, Muneera and Grundy, John},
2255 title = {Requirements Practices and Gaps When Engineering Human-Centered Artificial Intelligence Systems},
2256}Attribution
arXiv:2303.02920v2
[cs.SE]
License: cc-by-4.0
