Creativity and Connectedness
Creativity is vital to mankind’s progress. Thus, it is important to have a good understanding of the factors that impact it. The recent academic paper “Impact of network structure on collective learning: An experimental study in a data science competition” discusses the relationship between creativity and network connectivity properties. The paper is written by Devon Brackbill and Damon Centola, and published in the peer-reviewed open access scientific journal PLOS ONE.
The degree of separation of two nodes is the minimal length of a path between them. The small-world phenomenon, discussed in class, is the idea that the degree of separation in social networks is typically small for almost any pair of people; for example, in Milgram’s experimental study the median degree of separation is 6. A network is said to have higher efficiency (than another network) if it has a smaller average degree of separation. The paper studies how efficiency in communication networks impacts problem-solving. Communication networks are vital to finding innovative solutions to complex problems in the workplace. This is the case especially when individuals must invest lots of time and resources to try out new ideas.
A prominent theory on this topic says that the more efficient a network is, the more creative and thus more successful it will be at solving hard problems. It relies on research which has shown that the higher the efficiency of a network, the faster news of new discoveries spreads. Also, a highly efficient network enables good coordination between colleagues. The key idea is that more communication amongst colleagues leads to better problem-solving. However, there is an opposing theory that high efficiency in networks can actually reduce the quality of solutions found. In any network, acceptable solutions of medium quality arise first. In an efficient network people learn fast about others’ solutions and conform. In a less efficient network some people continue working on novel ideas, and eventually find better solutions. Thus, less efficiency provides more room for exploration, and this increases the chance of finding groundbreaking solutions. So, on the one hand, there is a popular theory which maintains that higher network efficiency facilitates better communication and leads to better solutions to complex problems. On the other hand, there is a theory which predicts that solutions of medium quality arise faster and spread, and thus discourage others from pursuing innovative and potentially better ideas. In addition, some doubts have been raised whether network efficiency has a clear impact on problem-solving.
Brackbill and Centola analyzed the relation between creativity and network efficiency. They ran experimentally designed Data Science competitions. The participants were data scientists and statisticians. Eight independent trials were conducted. The test design faithfully represents a real world problem-solving environment and makes it possible to measure the quality of the produced solutions. In particular, it solves the problem, encountered in previous research, that it is often difficult to differentiate between popularity and quality of solutions. Here is a description of the experimental design: In each trial, the participants are randomly divided into two competing groups. The only difference between the groups is the structure of their collaboration networks. One of the groups is maximally efficient, namely, every two individuals are connected. In the other group, every node has four neighbors. In the first seven trials, each group consists of 10 people; in the last trial, each group consists of 20 people. Both groups are given the same problem. Each trial comprises 15 rounds. Every round has the following structure: First, participants work on the problem individually. Then everyone gets to see 4 solutions. The people in the efficient group see the four best solutions found by the group. In the other group, everyone gets to see the solutions of their four neighbors. Every participant can then choose to pick a solution or continue looking for a new one.
The results show that on average, the best solution found by the inefficient groups was 20% better than the best solution found by the efficient groups. The members of the efficient groups quickly used the best initially discovered solution, temporarily making their average performance much better than that of the inefficient groups. Namely, 64% of the members in the efficient groups instantly copied the best available solution in the first round of a competition. However, as time went on, the members of the inefficient networks discovered better solutions, leading them to getting a better average solution. This happened because some members of the inefficient groups didn’t see satisfactory solutions in the early rounds, so there were higher incentives to keep trying new ideas.
The experiments show that the less efficient networks consistently got better solutions. Both the quality of the best solution found by the group and the group’s average solution quality were higher. Importantly, the efficient groups never discovered the optimal solution, whereas the inefficient groups found it in 50% of the trials. Thus, the paper demonstrates the importance of balancing the tradeoff between quick dissemination of ideas throughout a problem-solving group and facilitating exploration.
Reference:
Devon Brackbill and Damon Centola, Impact of network structure on collective learning: An experimental study in a data science competition, PLOS ONE, September 4, 2020.
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0237978