Research to improve software practice
The recent research “Benefits and Limitations of Job Rotation in Software Organizations: A Systematic Literature Review” published in the 20th International Conference on Evaluation and Assessment in Software Engineering (EASE 2016) has received a best paper award in the conference which was venued in Limerick, Ireland.
Next week two PhD students of HASE Research Group will participate of the 9th International Symposium on Empirical Software Engineering and Measurement that will be held in Beijing, China.
Our group has an accepted paper at the Conference and the presentation will discuss the current state of the replication work of empirical studies performed in software engineering research. These study is an update and an extention of the results produced in a previous mapping study published in 2011.
The paper will be presented during the first day of presentations in the session “Community Building & Infrastructure” and a preprint of the work is available to download on the ESEM program.
The list of publications was updated with the most recent articles produced by the HASE research group. Check them at haseresearch.com/publications.
We updated the list of publications with the most recent journal articles produced by the HASE group. Check them at haseresearch.com/publications.
The article “An Empirical Study on the Use of Team Building Criteria in Software Projects”, presented at ESEM2011 in Banff, Canada, received the joint Best Paper Award. The article can be found at 10.1109/ESEM.2011.14 and was co-authored by Fabio Q. B. da Silva, A. César C. França, Tatiana B. Gouveia, Cleviton V. F. Monteiro, Elisa S. F. Cardozo, and Marcos Suassuna.
The seminar I delivered at BYU on January 19th, 2012, can be seen at here. I summarize most of the results the HASE group produced over the past 6 years, and try to organize them in a coherent form.
Fabio Silva, Toronto, January, 2012.
The definition of effectiveness is important because what affects effectiveness (positive and negatively) depends on this definition, as well as the identification of factors that moderate or mediate the effects.
Our first studies on antecedents of effectiveness were on motivation (2007…). The “level of effort applied to tasks” is proposed as an antecedent of effectiveness by Hackman (1987) and supported in Yeatts and Hyten’s (1998) model, and this level of effort is driven by motivation. These models hypothesize that the more motivated the individual, more effort he or she applies to the task. Therefore, understanding motivation was an important step to understand effectiveness. Using a set of 20 motivators presented in a systematic literature review developed by Sarah Beecham and colleagues (Beecham 2008), we conducted a survey with software engineers from industry to identify the influence of each motivator on individual motivation. We grouped the motivators using principal component analysis and arrived at results that fit quite well on Hackman’s model (Hackman, 1987). We used these results to support the proposition of guidelines to develop motivational programs.
In 2009, we started a study about antecedents of success in Agile development. We focused that study on SCRUM projects. Our goal was to correlate the use of agile practices, as proposed as success factors by Chow and Cao, and the success of projects that used SCRUM. Our results (reported here and summarized here), showed that, in the particular context studied, only a subset of the success factors proposed by Chow and Chow correlated to project success.
We also investigated another type of antecedents that are related to team composition. In 2008, we started a mix method studie involving qualitative and quantitative research, to identify the criteria used by project managers to select members for software teams. We also correlated the level of use (formal, informal, not used) of these criteria with project success (a type of team effectiveness). The results, published last year at ESEM’2011, showed that the use of the criteria correlates with project success. That is, teams that were built using the set of criteria in a rigorous way were those that achieved more success.
So, at this point, we think we probably know (0OI) a couple of results things antecedents of effectiveness. However, we also know that we don’t know a lot of other things (1OI). For instance,
. how is motivation related to effectiveness? in other words, how is motivation related to all facets of effectiveness, including productivity, team member satisfaction, team continuity, etc.?
. what are the contextual factors that made only a few success factors correlate to project success in our SCRUM study? Are Chow and Cao wrong? We don’t think so. What is more likely is that (1) the success factors depend on contextual factors (moderators and mediators) that were not properly addressed because we did not know about them (2OI) and/or (2) the success factors affect different aspects of effectiveness that were not measured because we used a very simple (maybe oversimple) operationalization of success.
. it seems clear (from our results and also from various other studies in group theory) that team composition affects effectiveness. What is less clear is how team composition affects effectiveness and how task type and development process moderates this effect.
I plan to show the results of the research described above and then elaborate more on the questions that we are investigating with respect to antecedents of effectiveness. At this point in the seminar, I hope to start to show the model we are consolidating (from other researchers) and testing (with our empirical studies).
Fabio Silva.
References
Beecham, S., et al., 2008. Motivation in software engineering: a systematic literature review. Information and software Technology, vol. 50. Elsevier, pp. 860– 878.
T. Chow; D. Cao, (2007) A Survey Study of Critical Success Factors in Agile Software Projects. The Journal of Systems and Software, n. 81, pp. 961–971.
Hackman J (1987) The design of work teams. Handbook of organizational behavior 315-342.
Yeatts DE, Hyten C (1998) High-performing self-managed work teams. Sage Publications, Inc.
One of the topics I would like to discuss is “Team Effectiveness”. This is a complex topic that has been studied from different perspectives in group research.
Hackman (1987) has argued that, for teams in organizations (i.e., in practice and not in a experimental lab setting), effectiveness is more complex than counting “right answers” or timing the development of a task. Hackman argues that certain team outputs can be difficult to quantify. Furthermore, teams in organizational contexts tend to work together for a much longer time than the participants in a laboratory experiment, and what happens to a team during the development of one project may affect the feasibility of this team working together in the future. In his model, Hackman defines effectiveness as a multiple construct composed of
1. Acceptance of team’s output or results by those that receive or review them (Client Satisfaction with results)
2. Maintained or enhanced capability of members of the team to work together in the future (Team Viability)
3. Team member’s satisfaction has not decreased the the team experience (Member satisfaction with teamwork)
Cohen (1993) asks the question of whether we consider a team that produces what expected as being successful even if the team members present low morale, are not satisfied with the work, come late to work, or the team turnover is right. In her model, Cohen also considers effectiveness as a multidimensional constructed composed of:
1. Performance: costs, productivity, quality.
2. Member attitudes: satisfaction, trust, commitment.
3. Withdrawal behaviors: absenteeism, turnover.
Yeats and Hyten (1998) acknowledge that effectiveness is multi-dimensional, but only performance as client satisfaction and team economic viability. Their argument is that using a more complex, multi-dimensional view of effectiveness would make a model less accurate, since each dimension is affected differently by other factors that influence team effectiveness.
In the seminar, I will present some of our studies in which distinct definitions of effectiveness were used. I would like to discuss the implications (practical and theoretical) of using simple or more complex definitions in understanding the effectiveness of software teams in practice.
Fabio Silva.
References
Cohen SG (1993) The Design of Effective Self-managing Teams. Advances in Interdisciplinary Studies of Work Teams. Vol. 1 Theories of Self-managing Working Teams 1:
Hackman J (1987) The design of work teams. Handbook of organizational behavior 315-342.
Yeatts DE, Hyten C (1998) High-performing self-managed work teams. Sage Publications, Inc.
Professor Charles Knudson gently invited me to present a seminar at the Computer Science Department at Brigham Young University in Provo, Utah. This will happen on January, 19th, 11:00 – 11:50. This is a great opportunity to discuss some research ideas about human aspects in software engineering and to start a closer collaboration with researchers from BYU.
I will try to develop the presentation incrementally in this blog. Comments are welcome.
Let’s start with the title. My choice at the moment goes like this:
“The orders of ignorance in our research about the influence of human aspects in software engineering”
This is inspired by an article from Phillip G. Armour, published at the COMMUNICATIONS OF THE ACM, October 2000/Vol. 43, No. 10. Armour describes what he calls the five levels of ignorance we face in software development, and I think these levels are quite good at describing our ignorance with respect to a lot of other things, including research. These levels are (extracted from Orders Of Ignorance):
It is very likely that our research about human aspects in software engineering has different levels of ignorance (all but the 4OI). Therefore, I think it will make an interesting seminar to talk about:
Fabio Silva.
The model proposed by Hackman (1987) for the design and development of effective work teams is one of the most influential in the research about team work. Central to the model is the concept of task design as one of the conditions that support effort from team members. The task design concept postulates that a work group would “work especially hard on its tasks” when five conditions are met. One of these conditions is usually referred as task identify, defined by Hackman as follows:
“The group task is a whole an meaningful piece of work, with a visible outcome.” (Hacman, 1987, pp. 324)
In trying to use Hackman’s model in the design of tasks for software teams, one faces the problem of understanding the meaning of task identity in software development. Several alternatives to operationalize this concept in the software development would have distinct implications for task and team design.
First, and the most obvious, alternative is to consider that the development of a whole system. It certainly meets the criteria of task identify as a whole piece of work, meaningful and with a visible outcome. Some software teams in industry are responsible for the development of an entire system, but not all. In fact, it is more likely that the majority of the software teams in industry do not work on an entire system, but on parts of a system.
On the other end of a possible spectrum of task designs, is the case in which the development is broken in phases, each one assigned to specific teams. Coding and testing conducted by different teams are typical examples. One could argue that even in this case it is possible to have certain degree of task identity in each task, but this is certainly different from the first case. In this cases, it is more difficult for individuals or even teams to have an understanding of “the big picture.”
In the middle, there is the case in which a team is responsible for a part of the system, and performs all (or most of the) tasks associated with that part: design, coding, testing, maintenance, etc. Again, certain degree of identity exists that is “smaller” than the first case and “larger” than the second.
Aninteresting research problem is to understand the relationship between different types of task design and the motivation (or demotivation) these designs have on software engineering teams. In particular, it seems relevant to understand task identity in the context of agile and traditional development, as well as in the context of open source development. It is likely that in the three context the “levels” of identity will vary and, thus, the level of motivation that the task design will have on software engineers will also be different.
We are currently working on this problema and comments are welcome.
Fabio Silva.
Reference
Hackman J (1987) The design of work teams. Handbook of organizational behavior 315-342.
HASE - Human Aspects in Software Engineering 