Recommendation systems-based software requirements elicitation process—a systematic literature review

Abstract

Requirements elicitation is one of the fundamental sub-processes of requirements engineering which is used to find the needs of stakeholders. There are several activities in this sub-process, i.e., identification of stakeholders and their requirements, software requirements prioritization, and analysis. Recommendation systems have been intertwined with the requirements elicitation process to predict the stakeholders’ requirements based on their preferences for functional and non-functional requirements. A number of systematic literature reviews (SLRs) have been carried out in the area of requirements elicitation. These SLRs do not support the applications of the recommendation systems during the requirements elicitation process. To deal with this issue, we present an SLR on recommendation systems-based software requirements elicitation processes, from 2009 to 2022, undertaking four research questions: (a) What are the different activities of the software requirements elicitation methods? (b) What are the applications of recommendation systems in the identification of the software requirements? (c) How the recommendation systems can facilitate the identification of stakeholders in the requirements elicitation process? (d) What are the ways to automate the selection of requirements elicitation techniques? The aim of this study is to identify the research gaps in the area of recommendation systems-based requirements elicitation processes and suggest future research directions.

Introduction

Requirements elicitation is the first sub-process of requirements engineering (RE) which is employed to understand and identify the needs of stakeholders. Several methods and techniques have been developed to elicit the needs of stakeholders like contextual methods, cognitive methods, traditional methods, and goal-oriented methods [1]. Various techniques have been applied to strengthen the requirements elicitation process like fuzzy logic, rough-set theory, and recommendation systems. Among these techniques, recommendation systems are widely used in the literature to deal with a large set of requirements. The aim of a recommender system is to identify the items of interest according to the needs of stakeholders based on their past information or preference relations by applying different kinds of filtering methods, i.e., content-based filtering, collaborative-based filtering, and hybrid-based filtering [2]. In the literature of RE [2], the following problems have been addressed by recommendation systems, i.e., (a) identification of stakeholders of a project, (b) elicitation of customer requirements or features of a system, and (c) requirements selection and prioritization. Different datasets have been used in the literature in which recommendation system is used for eliciting the needs of customers, i.e., Replacement Access, Library, and ID Card (RALIC) [3] and Institute Examination System (IES) [4]. For example, Lim and Finkelstein [3] proposed a hybrid approach for eliciting a large set of requirements using social networks and collaborative filtering techniques. The applicability of this approach was discussed by considering the dataset of RALIC. This dataset contains 76 stakeholders and 104 requirements. Hassan et al. [4] focused on analyzing the stakeholders using confidence value where linguistic variables were used during the recommendation of the stakeholders. The authors have used the “canonical representation of multiplication associated with the ${{\text{L}}}^{-1}{{\text{R}}}^{-1}$ inverse arithmetic principle with graded mean integration representation” for selecting stakeholders based on the recommendations by other stakeholders for developing an IES. The RALIC dataset was also adopted by Shambour et al. [5] for dealing with one of the issues of requirements elicitation, i.e., information overload. In their work, the authors have developed a hybrid user-item-based method for eliciting the requirements using collaborative filtering.

There are several activities in the requirements elicitation process like stakeholder identification and their requirements, prioritizing the software requirements (SRs), etc., so that the product can be developed in accordance with the needs of the stakeholders. Improper elicitation of stakeholders’ requirements, lack of clarity of goals, and wrong cost estimation are the prominent reasons for a failed or challenged software system. According to the CHAOS report published in 2020, 19% of the projects failed while the outcomes of half of the projects were challenged and 31% of the projects were successful. One of the reasons for the failures of software was inefficient SRs’ identification and their prioritization [6]. Few studies have used recommendation systems to automate the different activities of requirements elicitation process. There are different types of recommendation systems that are primarily classified on the basis of their recommendation pattern, e.g., collaborative filtering recommendation systems, content-based filtering recommendation systems, and hybrid recommendation systems [7]. These systems are defined as follows:

• Collaborative filtering recommendation system: It provides recommendations on the basis of the prior preferences of similar stakeholders. For example, consider two similar stakeholders ${S}_{1}$ and ${S}_{2}$, both selecting requirements ${R}_{1}$ and ${R}_{2}$ for a software product. If stakeholder ${S}_{1}$ selects another requirement ${R}_{3}$ for the software product, then ${R}_{3}$ will be recommended to ${S}_{2}$ as well; see Fig. 1. Collaborative filtering is the prominent technique that is employed in recommender systems. Such recommender systems are dependent on two important computations, i.e., computing similarity and rating prediction. There are different techniques that are used in literature for computing similarity, e.g., Pearson’s correlation coefficient, cosine similarity, adjusted cosine similarity, and Euclidean distance [7]. Equation 1 describes the formula to compute the similarity between the two stakeholders${S}_{1}$ and ${S}_{2}$ using Pearson correlation, i.e. ($stakeSim\left({S}_{1},{S}_{2}\right)$).

  $$stakeSim\left({S}_{1},{S}_{2}\right)= \frac{{\sum }_{R\in {CR}_{{S}_{1},{S}_{2}}}\left({r}_{{S}_{1}R} - {\overline{r} }_{{S}_{1}}\right)\left({r}_{{S}_{2}R} - {\overline{r} }_{{S}_{2}}\right)}{\sqrt{{\sum }_{R\in {CR}_{{S}_{1},{S}_{2}}}{\left({r}_{{S}_{1}R} - {\overline{r} }_{{S}_{1}}\right)}^{2}}\sqrt{{\sum }_{R\in {CR}_{{S}_{1},{S}_{2}}}{\left({r}_{{S}_{2}R} - {\overline{r} }_{{S}_{2}}\right)}^{2}}}$$

  (1)

Fig. 1

Illustration of collaborative filtering recommendation system

Here_, ${CR}_{{S}_{1},{S}_{2}}$ = Set of co-rated requirements between two stakeholders ${S}_{1}$ and ${S}_{2}$

• ${r}_{{S}_{1}R}$: Rating given by stakeholder ${S}_{1}$ to requirement R

• ${r}_{{S}_{2}R}$: Rating given by stakeholder ${S}_{2}$ to requirement R

• ${\overline{r} }_{{S}_{1}}$: Average rating of stakeholder ${S}_{1}$

• ${\overline{r} }_{{S}_{2}}$: Average rating of stakeholder ${S}_{2}$

Values generated by Eq. 1 lie between “1” and “−1”. The perfect agreement between the stakeholders ${S}_{1}$ and ${S}_{2}$ is represented by value “1”; on the other hand, “−1” represents the value of perfect disagreement between them. This process is repeated to compute the pair-wise similarities between different stakeholders. Further, a neighborhood is generated for each stakeholder by choosing “N” for most similar stakeholders. The interest level of stakeholders, e.g., ${S}_{1}$, for requirement “R” which ${S}_{1}$ has not yet rated is predicted by the following equation:

$$pred\left({S}_{1},R\right)= {\overline{r} }_{{S}_{1}}+ \frac{{\sum }_{{S}_{2}\in nbr({S}_{1})}stakeSim\left({S}_{1},{S}_{2}\right)\cdot ({r}_{{S}_{2}R} - {\overline{r} }_{{S}_{2}})}{{\sum }_{{S}_{2}\in nbr({S}_{1})}stakeSim\left({S}_{1},{S}_{2}\right)}$$

(2)

Here ${S}_{2}\in nbr({S}_{1})$: ${S}_{2}$ is a neighbor of ${S}_{1}$

• Content-based filtering recommendation system: It generates recommendations on the basis of the past preferences of a stakeholder [7]. For example, if a stakeholder ${S}_{1}$ selects a requirement ${R}_{1}$ for a software product, then requirements similar to ${R}_{1}$ (e.g., ${R}_{2}$) will be recommended to ${S}_{1}$, as illustrated in Fig. 2. Content-based filtering in requirements elicitation requires representation of requirements, stakeholder profiling, and calculating similarity measure for generating recommendations. Each requirement ${R}_{i}$ is mathematically represented as a feature vector ${V}_{i}$. Stakeholder profile ${P}_{S}$ is then created after a combination of the feature vectors of the stakeholders’ requirements in which weights are assigned based on stakeholders’ preferences; see Eq. (3).

  $${P}_{S}={\sum }_{i\in {{\text{stakeholders}}}^{\mathrm{^\prime}}\mathrm{ \,requirements}}{w}_{S,i} . {V}_{i}$$

  (3)

where ${w}_{S,i}$ is the weight assigned to the requirements based on stakeholders’ preferences.

Fig. 2

Illustration of content-based filtering recommendation system

Similarity measure is computed between stakeholder profile ${P}_{S}$ and each requirement’s feature vector ${V}_{i}$. Equation (4) represents the formula to calculate cosine similarity between ${P}_{S}$ and ${V}_{i}$.

$$Cosine\_sim\left(S,i\right)= \frac{{P}_{S} . {V}_{i}}{{||P}_{S}|| . {||V}_{i}||}$$

(4)

where ${||P}_{S}||\, {\text{and}}\, {||V}_{i}||$ are the Euclidean norms of vectors ${P}_{S} \,{\text{and}} \,{V}_{i}$ respectively.

Based on the similarity values, the requirements are ranked signifying the corresponding relevance during requirements elicitation.

• Hybrid recommendation system: It generates recommendations by amalgamating the features of both collaborative and content-based recommendation as per the requirements of the product [7].

Various SLRs have been conducted to investigate the strength and weaknesses of the existing requirements elicitation techniques. For example, Pacheco and Garcia [8] conducted an SLR on the stakeholders’ identification methods applied in requirements elicitation. Consequently, it was found that the methods based on stakeholders’ identification have been categorized into the following: (a) methods describing stakeholders, (b) methods based on the interaction between the stakeholders, (c) methods assessing the stakeholders. The assessment of the stakeholders depends on the priority interest and the skills of the stakeholders in the project development. One way of getting the expected quality of software product is to understand the environmental domain where the software project is expected to be developed. An efficient and effective stakeholders’ selection results in an improved coverage of requirements [8]. Hujainah et al. [9] performed an SLR on the significance of stakeholders and techniques in requirements prioritization process. The authors have categorized the requirements prioritization techniques based on the manual, semi-automated, and fully automated types. Aldave et al. [10] investigated the requirements elicitation techniques within agile software development. To date, the most recent SLR is on the data-driven requirements elicitation performed by Lim et al. [11] emphasizing automated requirements elicitation for information systems. A list of selected SLRs in the area of SRs elicitation is summarized in Table 1. To our knowledge, no SLR has been performed with an emphasis on the applications of recommender systems in requirements elicitation. Thus, the objective of this study is to perform an SLR in the area of recommendation systems-based requirements elicitation so that key stakeholders and their requirements can be identified and recommended during the requirements elicitation process.

Table 1 Selected SLRs in the area of SRs elicitation

The subsequent sections are organized as follows: The “Research methodology” section discusses the research methodology for conducting the SLR. An insight into the threats to validity is described in the “Threats to validity” section. The results and discussion based on the research questions (RQs) are summarized in the “Results and discussion” section. A comparative study between the SLR on the recommendation systems-based requirements elicitation process and other selected SLRs related to requirements elicitation processes is performed in the “Comparative study” section. And in the end, the conclusion, challenges, and future work are summarized in the “Conclusions, challenges and future work” section.

Research methodology

The guidelines proposed by Kitchenham and Charters [14] are applied to identify the research gaps in the area of recommendation systems-based SRs elicitation process. In accordance with [14], the following steps are taken to perform the SLR: (a) research questions, (b) search strategy, (c) study selection, and (d) data synthesis. A detailed description about these steps is given below:

Research questions

The objective of recommendation systems-based methods for the elicitation of SRs is to capture the needs and preferences of the stakeholders so that different types of SRs can be identified in global software development according to the needs of the stakeholders. These stakeholders are placed at different locations in a country or abroad. To achieve this objective, the following RQs are formulated:

• RQ-1: What are the different activities of the SRs elicitation methods?

• RQ-2: What are the applications of recommendation systems in the identification of the SRs?

• RQ-3: How the recommendation systems can facilitate the identification of stakeholders in requirements elicitation process?

• RQ-4: What are the ways to automate the selection of requirements elicitation techniques?

Search strategy

The following keywords have been derived from the above RQs:

• “software requirements elicitation,” “recommendation systems,” “stakeholder,” “automation,” “elicitation technique.”

A search string was constructed based on the above keywords. The synonyms of these keywords were also made to complete the string so that relevant studies based on the RQs can be identified. The terms that appeared in the keyword were expanded using Word Net Version 3.0 [15] and Oxford Dictionary (https://www.oxfordlearnersdictionaries.com/) of English synonyms. Finally, the following search string was created to identify the primary studies from the electronic databases:

• Search string: ((Software requirements elicitation OR Software requirements engineering OR Requirements elicitation OR Requirements engineering) AND (Recommendation system OR Recommender system OR Automation OR Limitations OR Weaknesses OR Strength OR Advantages OR Disadvantages) AND (Review OR Systematic review OR Literature review OR Systematic literature review OR Survey OR Journey OR Literature mapping OR Systematic literature mapping OR State-of-the-art)).

We used the following five electronic databases to search the relevant primary studies based on the search string: IEEE Xplore, ACM digital library, Springer, ScienceDirect, and Google Scholar.

Study selection

The search strategy for the study selection process is exhibited in Fig. 3. We have selected the primary studies published from 2009 to 2022; see Fig. 4. Initially, 170 primary studies were selected from the five electronic databases. After being subjected to scrutiny on the basis of title, 60 of the studies were unlisted from the SLR as they were unnecessary and irrelevant. As a result, 110 primary studies were selected based on the title. These studies were further analyzed on the basis of the abstract and conclusion and 60 studies were shortlisted.

Fig. 3

Search strategy for the study selection process

Fig. 4

Year-wise distribution of primary studies

The selected studies were finally assessed on the basis of the following quality assessment (QA) criteria:

• QA-1: “Do the selected studies serve the purpose of answering the RQs?”

• QA-2: “Is the aim of the research clearly conveyed?”

• QA-3: “Is there any case study to support the research?”

• QA-4: “Does the research add any value to industry or academia?”

A grade point of 0.5 is assigned for a QA question when a study partially answers the question. When the study satisfactorily answers the QA criteria, it is assigned a grade point of 1.0. For every selected study, the sum of the grade points corresponding to the QA criteria is evaluated. If the sum is greater than or equal to 2.0, the study is selected as a primary study for the SLR. Finally, 50 studies were identified as primary studies and selected for the SLR. A list of the selected 50 primary studies is summarized in Appendix 1 and the quality assessment scores of these studies are presented in Appendix 2. The primary studies include research papers from the following journals and conferences of international repute, e.g., “IEEE Transactions on Software Engineering,” “Requirements Engineering Journal,” “Conference proceedings published by LNCS,” “International Conference on Software Engineering,” and “International Conference on Requirements Engineering.” The formulated RQs have been answered by considering the selected 50 primary studies.

Data synthesis

The data related to the 50 primary studies are synthesized for answering the formulated RQs. The following ways are adopted to synthesize the data: The answer to the RQ-1 is depicted by a bar graph to illustrate the various activities carried out during SRs elicitation methods; see Fig. 5. The answer to the RQ-2 is tabulated in Table 2, which summarizes the different activities of SRs elicitation where recommendation system facilitates the identification of SRs. The result for RQ-3 is represented in a tabular form (see Table 3), which depicts various techniques employed by recommendation systems to automate the identification of stakeholders in requirements elicitation process along with the level of automation and the scale of the software project under consideration. The observations to the RQ-4 are documented in the form of text.

Fig. 5

Different activities in requirements elicitation methods

Table 2 Recommendation system in SRs elicitation activities

Table 3 Scale of project, techniques and level of automation in stakeholders’ recommendation process

Threats to validity

The aim of this section is to discuss the potential issues that can affect the conclusion of an SLR. There are four major threats to validity in any SLR, “conclusion validity,” “internal validity,” “construct validity,” and “external validity” [16]. These threats are an essential part of any SLR. For example, Sadiq et al. [16] discussed these threats for selecting the SRs with incomplete linguistic preference relations. A brief discussion about different types of validation is given below:

• i. One of the key threats to the conclusion validity is the biasness in the selection of appropriate primary studies and the synthesis of data. To weaken this threat, a systematic study selection approach was designed for the inclusion and exclusion of the primary studies based on the QA criteria. This approach was enforced accurately to confirm the correctness of each included primary study.

• ii. In internal validity, the relationship between the variables of interest and the results is discussed. In this study, the applications of the stakeholders during the requirements elicitation and prioritization are discussed in the context of the recommendation systems. Few methods have been applied only for the small datasets of SRs with a main focus on the local software development rather than considering the global software development. As a result, the right performance of the methods to deal with large projects using recommendation systems might not be evaluated. To alleviate this issue, multiple sources of publications of the same work have been considered, whenever possible.

• iii. The relationship between the application and theory is discussed in construct validity. One of the threats to the construct validity arises from the elimination of possible related primary studies. To curb this threat, the search procedure was defined and all the studies related to the recommendation systems-based SRs elicitation and prioritization domain were included. The grey studies related to the domain were excluded.

• iv. Finally, the threats to external validity include the questions that limit the capability to generalize the findings of SLR outside the scope of the study. In this study, the grey and non-English studies are excluded. We believe that the review protocol used in this study helped us to choose a typical set of studies related to domain knowledge. The results of this study are more concerned with the requirements elicitation using recommendation systems domain from the industrial as well as the academic perspectives.

Results and discussion

In this SLR, the 50 primary studies are identified based on the review protocol. These studies have been published from 2009 to 2022. The year-wise distribution of the primary studies is depicted in Fig. 4.

RQ-1: What are the different activities of the SRs elicitation methods?

Elicitation is the group of activities that helps in gathering information for a specific problem domain [S1, S2], and errors occurring in this phase hamper the success of the proposed software system [S3, S4]. The requirements elicitation, carried out by following group of activities, is governed by different factors like scope and objectives of the project, type and place of the organization, and scale of the system. These factors enhance the clarity, consistency, effectiveness, and unambiguity of the requirements [S5, S6].

Based on our analysis, it is found that some of the activities are common in requirements elicitation methods. For example, Pohl [S7] described three broad categories of the requirements elicitation activities that are commonly performed. Sandhu and Weistroffer [S8] highlighted the influence of requirements elicitation in the fulfillment of the needs of the organization and the stakeholders. The authors listed five primary tasks in requirements elicitation and comprehensively reviewed the importance and the challenges linked with the tasks. Mulla and Girase [S9] proposed a compendious study and focused on five different activities for the requirements elicitation. Sharma and Pandey [S10] discussed the need for a more detailed description of the requirements elicitation. In their work, the authors have considered ten activities of the requirements elicitation and identified various concerning challenges associated with them. Bani-Salameh and Aljawabrah [S5] also proposed 10 different activities in their requirements elicitation model. The authors through their model tried to generate all the correct requirements for a given software project. Jalil et al. [S11] divided the requirements elicitation processes into 5 phases. Wong and Mauricio [S12] proposed seven different activities embedded in the requirements elicitation process. The authors also identified the factors influencing these elicitation activities. The “software engineering body of knowledge” (SWEBOK) [S13] has proposed two activities that are considered the most important in SRs elicitation.

On the basis of our review, Fig. 5 summarizes the requirements elicitation activities which are common in the existing elicitation methods. From Fig. 5, it is clear that the following activities are common in most of the requirements elicitation techniques: identifying the application domain, stakeholder identification, and analysis and identifying the sources of requirements, documentation, and refinement. In addition to these activities, some activities have also been introduced to handle vagueness and imprecision in the elicitation methods. For example, Sadiq [S14] proposed a fuzzy-based approach for stakeholders’ analysis so that key stakeholders can be identified based on the importance of SRs. Among various SRs elicitation techniques, goal-oriented methods have also been given due importance. In such methods, the goals of the stakeholders are broken down into sub-goals to get the functional requirements (FRs) and non-functional requirements (NFRs) of the system [S15]. Mohammad et al. [S16] developed a fuzzy-based method for SRs analysis in the goal-oriented domain. To address the issues of fuzzy-based methods, Sadiq and Devi [S17] proposed a method using rough-set theory for the prioritization of SRs. In another study, Sadiq and Devi [S18] developed a method for the selection of requirements of an IES using fuzzy-soft set approach. In these studies, small and medium datasets have been applied for the explanation of the proposed methodologies. Amaral and Elias [S19] proposed a risk-driven multiple objective evolutionary method for SRs selection. In a recent study, Nazim et al. [S20] discussed different types of datasets used in SRs selection and prioritization research. In their study, fuzzy AHP and fuzzy TOPSIS have been compared based on the datasets of an IES.

RQ-2: What are the applications of recommendation systems in the identification of the SRs?

There are different activities associated with the requirements elicitation process as shown in Fig. 5. These activities are very essential to gather the correct requirements from different stakeholders using different techniques. When performed manually, these elicitation activities can be extensively time-consuming and prone to errors [S21]. Recommendation systems help in solving this problem.

In requirements elicitation methods, recommendation systems are used to recommend the prospective stakeholders. The recommendation systems keep the stakeholders updated by providing information about the project. The consensus among the stakeholders can also be achieved through the recommendation systems if the points of view of the stakeholders are efficiently considered [S22, S23].

Dumitru et al. [S24] proposed a recommendation system that deploys an incremental clustering approach for domain analysis. The approach emphasized the specifications of the available software products to recommend features that are viable for the software product under development using Association Rule Mining (ARM) and k-Nearest-Neighbor (k-NN). There are some studies in which recommendation systems have been combined with social networks so that the requirements of large-scale project can be identified. For example, Lim et al. [S25] developed a StakeNet methodology for eliciting a large set of SRs. The authors have employed social network measures for automating stakeholder analysis so that stakeholders can recommend each other as per the necessities of the project. One of the key steps of the StakeNet methodology is to collect profiles of the stakeholders and gather information in order to generate a prioritized list of the SRs.

Various other studies have also discussed the applications of recommendation systems during the requirements elicitation process. For example, a context-aware recommendation system proposed by Roher and Richardson [S26] addresses activities like exploring the application domain, identifying the goals of the organization, and channelizing these goals. The proposed recommender system also takes into consideration the place of deployment of the project to assist in integrating sustainability. Ninaus et al. [S27] developed an approach to assist elicitation activities that includes supporting stakeholders’ identification, prioritization and quality assurance of requirements, requirements reuse, and planning the software release. This approach is referred to as INTELLIREQ, which utilizes the advantages of different recommendation techniques, e.g., content-based recommendation using Dice coefficient, group recommendation using Majority voting, knowledge-based recommendation using preference matrix, etc., to make the requirements model more consistent and proactive. Iqbal et al. [S28] discussed the effects and applications of machine learning to automate the different requirements engineering tasks, e.g., requirements elicitation and discovery and requirements specification. The authors pointed out that machine learning provides better decision-making for a software project dealing with a large dataset with high degree of imprecision and ambiguity. Ahmad and Sadiq [S29] proposed a recommendation systems-based approach for prioritizing the requirements of an IES. In their work, the authors have identified a list of stakeholders and their requirements. Consequently, the elicited FRs and NFRs were selected by using the “${{\text{L}}}^{-1}{{\text{R}}}^{-1}$ inverse function arithmetic principle and graded mean integration” representation. Lunarejo [S30] has proposed a semi-automatic multi-criteria approach to address scalability and lack of automation in requirements identification for FRs and NFRs of software products. The proposed approach has been evaluated using real web-based geographic information systems (GIS). Mohebzada et al. [S31] in their systematic mapping focused on the applications of the recommendation systems in recommending stakeholders, priority of requirements, similar requirements, etc. It was observed that collaborative filtering has been used to generate the recommendations.

From the literature, it is evident that recommendation systems play a prominent role during requirements elicitation process and the automation tools and techniques based on artificial intelligence, and machine learning has received much attention during the requirements elicitation process. Table 2 summarizes the different activities of SRs elicitation where the recommendation system facilitates the identification of SRs. But simultaneously there are certain challenges of recommender systems as well during the elicitation activities, e.g., different types of stakeholders’ feedback and relationships between requirements [S32].

RQ-3: How the recommendation systems can facilitate the identification of stakeholders in requirements elicitation process?

People from different domains are engaged in requirements elicitation for a software project. These individuals or group of individuals are the stakeholders. The success and the failure of software project are influenced by the involvement of varied stakeholders [S9]. Identifying the stakeholders has always been intensive during the implementation and requires great deal of effort and time to finalize the complete list of stakeholders for a software project. Recommendation systems facilitate these tasks and collects information that are valuable for the proposed software system [S36, S37] as depicted in Fig. 6. Thus, many studies have focused on simplifying and automating the stakeholders’ identification process. Mulla and Girase [S9] in their study have identified various social network measures that can be used for the recommendation and also for the prioritization of stakeholders. The study suggested to build a social network whose nodes depict stakeholders. The links of the network are the stakeholder’s recommendations. Here, the stakeholders recommend other stakeholders for the identification and prioritization. Castro-Herrera et al. [S38] analyzed the downside of the traditional techniques in large software projects with substantial number of stakeholders. The authors further highlighted the significance of the recommendation system in facilitating the recognition of key stakeholders.

Fig. 6

Stakeholders recommendation process in a software project

In [S38], the authors have suggested a hybrid recommendation system to identify potential users to solve the unattended threads in open-source forums. Lim et al. [S25] developed StakeNet, which identifies, recommends, and prioritizes the stakeholders associated with a software project using social networks. Hujainah et al. [S39] proposed a new semi-automated technique termed StakeQP that facilitates stakeholder quantification and prioritization during software requirements elicitation. This method was evaluated by using the RALIC dataset to show its relevance and effectiveness in facilitating the recommendation of stakeholders and the subsequent requirements. Palomares et al. [S40] developed the OpenReq approach to propose recommendations of relevant stakeholders for the requirements elicitation. The approach utilizes the collaborative filtering recommendation mechanism by examining the contributions of the stakeholders in the earlier software projects, analyzing the strength and the interest of the stakeholders in a software project domain and personal availability of the stakeholders. Hariri et al. [S41] in their work suggested the identification and recommendation of stakeholders with proficiency in the software project under consideration. The authors have proposed the implementation of a hybrid recommender system to recommend three categories of stakeholders, i.e., direct stakeholders, indirect stakeholders, and inferred stakeholders. The work by Castro-Herrera et al. [S42] concentrates on identifying and bringing together different stakeholders into relevant online forums and discussion groups. The authors have incorporated both data mining and machine learning in their framework for providing semi-automated assistance to manage these requirements forums. To identify and recommend potential experts for a domain, the approach by Castro-Herrera and Cleland-Huang [S43] automatically examines the contribution of different stakeholders. The approach then uses machine learning to identify, examine, and categorize these contributions into different domains. The profiles of the stakeholders thus created help in classifying the stakeholders. Milano et al. [S44] in their analysis of multi-stakeholder recommendation systems suggest the conclusive advantages of multi-stakeholder approach over the traditional user-centric perspective of recommendation systems. The authors have highlighted and suggested to study the impact of recommendation systems on the benefits of different stakeholders. The authors have also credited the role of recommendation systems in facilitating the interactivity of a large number of stakeholders in online forums. Felfernig et al. [S45] in their study observed a growing need for smart software systems to automate the support to stakeholders. The authors have channelized the importance of social networks in the identification and recommendation of stakeholders and clustering of SRs for identifying the dependency among them.

Based on our review, we observed that stakeholders, being a prominent factor in the success of a software project, have received due consideration in the recommendation systems. Researchers and academicians have put forward various models to automate the activities pertaining to the stakeholders, i.e., identification and analysis of stakeholders, recommending stakeholders, providing recommendations to stakeholders, etc. Stakeholder identification methods have mainly focused on different types of projects, techniques used for its recommendation, and levels of automation; see Table 3.

RQ-4: What are the ways to automate the selection of requirements elicitation techniques?

The requirements elicitation techniques are employed to examine the stakeholders’ needs in determining the requirements of the software under consideration. These techniques may be grouped into traditional techniques, group elicitation techniques, cognitive techniques, contextual techniques, goal-oriented techniques, quality function deployment methods, package-oriented requirements elicitation, etc. One of the challenges which is faced by facing requirements’ engineer during SRs elicitation is the selection of an appropriate elicitation technique [S46].

Darwish et al. [S47] listed a variety of factors that decide the techniques to be selected for a given software project. These factors include the level of criticality of the project under consideration, size of the project, and degree of project complexity. In their work, a hybrid machine learning model to select elicitation techniques has been proposed using 3-component approach. The approach begins with a literature review to identify common elicitation techniques and the factors affecting them. The second step is the identification of the factors affecting the technique selection using a multiple regression model. Finally, the required elicitation techniques are selected using a proposed Artificial Neural Network model. Tiwari et al. [S48] focused on selecting the methods for SRs elicitation. The authors underlined how the lack of recommendation system to select the elicitation techniques forced the stakeholders to use traditional company practices or individual experience. Hussein et al. [S46] in their work listed 19 factors which have been categorized into 4 broad categories, i.e., elicitor, stakeholder, project, and elicitation process that influence the specification and selection of requirements elicitation techniques. The authors have developed a prototype to assist the requirements engineers in performing the task. While most of these selections are done manually using the expertise of the requirement engineers, Ibrahim et al. [S49] proposed a model to automate the process. In their proposed model, the authors have used machine learning approach to automate the selection of elicitation technique. The authors have used k-NN algorithms to select the most appropriate technique to assist the requirement engineers in planning for the new project accounting for the requirements complexity characteristics. Moreover, Dafaalla et al. [S50] proposed a deep learning-based decision-making model for automating requirements elicitation technique selection. The authors, through their model, intend to reduce human errors thus enhancing the efficiency of the requirements elicitation in the development of a software project.

Based on our review, it is found that very few studies have focused on the automation of the requirements elicitation techniques selection. In most of the studies, theoretical and heuristic approaches have been explored for selecting elicitation techniques for the project under consideration. But few recent studies have proposed machine learning models to automate the requirements elicitation techniques selection.

Comparative study

In this section, we have compared the SLR between the recommendations systems-based SRs elicitation process and other selected methods based on the following criteria: area of SLR, year of publication, RQs, and support of recommendation system. The result is exhibited in Table 4. On the basis of the comparative study, it is found that requirements elicitation is a key process of software development. There are different aspects of requirements elicitation like stakeholders’ identification [8], maturity of the requirements elicitation techniques [1], application of data-driven concepts in requirements elicitations [11], and SRs selection [16]. Different issues related to the requirements elicitation techniques have been discussed in the existing SLRs [1, 8,9,10,11,12,13]. We could not find any relevant study from 2009 to 2022 that presents an SLR in the area of recommendation systems-based requirements elicitation process. Therefore, in this paper, an attempt has been made to fill this research gap.

Table 4 Comparative study

Conclusions, challenges and future work

This paper presents an SLR of the recommendation system-based SRs elicitation process. In this SLR, a review protocol was constructed to formulate the RQs. The results of this SLR present the different activities involved in the identification of the SRs, applications of the recommendation systems during the SRs elicitation process, and also the identification of the stakeholders, and finally, it discusses the ways to automate the selection of SRs elicitation techniques.

For SRs elicitation process, elicitation activities are indispensable for selecting a comprehensive, complete, and consistent list of requirements for project under consideration. Our SLR finds that identifying the application domain, stakeholder identification, identification of the sources of requirements, stakeholder analysis, and selection of tools, techniques, and approaches for elicitation are the prominent elicitation activities that have been used in real-life applications. Most of these elicitation activities were performed manually in the form of brainstorming, questionnaires, discussions, meetings, etc., but in recent times there has been an incremental rise in automating the tasks. Thus, our SLR further underlines the importance of various recommendation techniques in requirements elicitation and in particular the recommendation and identification of stakeholders and their requirements.

Collaborative recommendation techniques and social network measures are the most implemented methodologies to automate the stakeholders’ recommendation process for a software project. On the other hand, less attention is given to recommendation-based approaches for analysis of risks, software cost estimation, requirements tracking, and identification of NFRs. The findings revealed that existing SRs elicitation techniques have some limitations and associated challenges. For example, for a large-scale system with a very large number of stakeholders or for system where stakeholders change their opinions very frequently, there may be discordances in the stakeholders’ requirements and subsequent goals. These discordances are addressed by negotiation between the stakeholders which are mostly performed heuristically. Categorization of these discordances and automating the negotiations to increase the quality of the software needs further research. Additionally, in goal-oriented requirements elicitation method, the main emphasis is on the analysis of SRs using AND/OR graphs. Integrating the recommendation systems with goal-oriented methods for analyzing the goals and stakeholders’ discordances is an important issue that needs to be addressed in the future.

Appendices

Appendix 1

Table 5 Primary studies

Appendix 2

Table 6 Results of the QA scores of the selected 50 primary studies