Archives for the month of: February, 2013

Today the lab group were asked to be lab rats for the first-time-in-the-world-ever playing of a computer game! We also welcomed a couple of guests: Helen Pain from the University of Edinburgh and John Thompson from the Institute for Development Studies

African Farmer is an online, multiplayer game that is being written by Ellie and Jim for the Green Revolution project. It is loosely based on a couple of boardgames used in teaching scenarios for International Development and for policy makers. The game is ideally aimed at groups of between 15 and 25, with players being asked to take care of a family in a rural farming village in Africa.

The game revolves around a series of locations that each provide different information and access to functionality:-

  • Home – gives information about the health and wealth of the family. 
  • Farm – shows the state of the crops and fields that the family own.
  • Village – allows the players to get a feel for how other people in the village are getting on.
  • Market – allows players to buy and sell things that they need, such as crops, fertiliser, food, school vouchers etc.
  • Bank – for loans, and the payment of penalties.
Farm view

The farm view.

The idea is that people have incomplete information about the growing conditions, how much their crop will be worth and so on, and have to make some quite complex decisions. The outcome for the families can be harsh – if they do not secure enough food each year, family members will die. The simulation is a very simplistic view of farming and environment, but the decisions soon mount up and allow for a wide range of strategies to be employed.

This was the first time that the game had been run with multiple people logging in simultaneously, and the lab group were asked not to hold back with their feedback (although Ellie did say if they wanted to be nice about it they could be!). We had a lively session which completely overran our usual 12pm stop, and generated a lot of feedback on the interface. Sadly we didn’t get quite as far through the game as was hoped. Some of the interface difficulties slowed down people getting to grips with the game and being able to find the right information.

Still, as Jim put it this was a pre-alpha version, and the feedback was immensely useful for clearly indicating which areas need more work. The range of backgrounds within the group highlighted different problems – some very quickly got to grips with the communication panel and other elements of the interface, others grasped the flow of the game much more quickly. The game itself actually fared quite well – the group were absorbed and wanted to see what the outcomes of their actions were – so with a few changes to the interface the whole thing is looking extremely promising. Jim and Ellie have been left with a lot of ideas to think over.

Maybe after the next iteration the lab group will agree to play again!

Ostensibly this meeting was to discuss the paper that Ben had forwarded around the group on Grounded Theory. Actually this paper was more of an entry point for discussion. Nick Collins (a colleague from Music Informatics) has recently started some qualitative research, and was looking to discuss some of his reservations about the methods that he was having.

In a way, this made starting from Grounded Theory an interesting choice. Whilst many of the group members have conducted qualitative research, none of the group who were present have actually used Grounded Theory itself. So although we did spend a little time discussing the area the more interesting discussion came when we started talking more specifically about Nick’s current research interest and problem, and how the group’s experience could help.

Nick is doing a series of semi-structured interviews, the questions for which came from the results of a previous online study. Judith suggested that perhaps he should look at Thematic Analysis as an approach, but Katy has had some trouble finding a good reference for how to go about that.

Lesley uses an iterative approach to coding, starting potentially with very low level coding (examples used were “red flower” and “my mum said” – which did make sense in the context, honest!) and use those to then create higher level categories (e.g. “External objects”, “other people”) and recode. She says she tries not to have any preconceived ideas about what she’s looking for, but accepts that on some level she does.

Bias (which is introduced by the preconceptions) came up repeatedly. It seems that different studies have quite different approaches to this problem. Some work around it by having a large number of coders and looking for a high degree of inter-coder reliability. Others use two coders, who only confer when there is a disagreement. Still others use a single coder (and it was a real shame that Gareth wasn’t here at this point) who provides a single clear viewpoint. As always, the problem is finding the approach that is most appropriate for each particular study.

A related question was when to start looking at the data. Nick was very worried about looking back on it too soon, in case it changed the way he conducted later interviews. Lesley had examples where tweaking the questions on the way through had uncovered important information that would otherwise have been missed, although the change had to be documented when writing up the study. She also said that when you carry out your own interviews to code you can’t help but know what’s been said previously. Judith queried about the difference between knowing what had been said and reflecting on it, forming further ideas about it. The group consensus seemed to be that leaving it as long as possible before that happens would be preferable – which may explain why none of us have used Grounded Theory!

All in all it was a good discussion of the practices that the group have used when conducting research in the past. We hope Nick found it useful and not too overwhelming. We also hope that he might find time to come back and tell us about what he finds.

Given that a few of the group are attempting to write up at the moment, this seemed like the perfect time to set up our own little “Shut up and write!” session.

If anyone wants to join us, we’ll be in the Bridge cafe, 2pm on Friday 15th. All welcome (tables permitting), not just those trying to write their thesis. At the moment, Pejman and Ellie are confirmed (links for photo id purposes).

We had a couple of CHI-related announcements today – Ellie has a workshop paper accepted for Designing and Evaluating Sociability in Online Video Games at CHI, and Pejman (who is already helping to run the Games User Research workshop and has a full paper) has a student with a work in progress paper accepted. Nice work!

Today the group was focussing on Masters students. Each student has to do a research project as a part of the qualification. These projects are supervised by a current member of faculty who hopefully shares some interest in the topic! Some students come up with their own topics, but each member of faculty also lists a number of potential projects in a database. Today the lab group spent the meeting talking through and exploring some potential projects for the database. 

One of the main problems with this kind of exercise is getting the size of the project right. Some projects were dismissed for being too simple, some for being too complex or needing too much time for studies. There was an extended discussion about the ethics procedure – most students will be using some human participants in a project in HCI! 

The technical ability of HCI students can also be quite difficult to accomodate. Some have come from a computer science background and would be very happy to write a piece of software. Others come from more of a psychology background, and having to write the software before they test it may make the project too daunting. 

In the end a range of options did come together, although further work is needed before they get listed in the database. It was a good opportunity to discuss areas for further research on a number of the group’s projects, as well as for some “blue sky thinking” about further options. 

At this weeks lab meeting we discussed the following paper, as suggested by Ellie:

Lim, S., & Reeves, B. (2010). Computer agents versus avatars: Responses to interactive game characters controlled by a computer or other player. International journal of human-computer studies, 68(1), 57-68.

Judith led the session, and gave a brief overview of the paper before opening up the discussion. The paper set out to examine how perceived co-player agency and competitive vs cooperative game activities influence player engagement. The authors used physiological and self-report measures to examine the following hypotheses:

  • H1. Players will exhibit greater physiological arousal when their co-player’s character is an avatar rather than an agent.
  • H2. Players will exhibit greater physiological arousal when they are engaged in a competitive gaming task rather than in a cooperative task.

There were many positive aspects about this paper, and there was general agreement that it was well-written and clearly structured. Many of us appreciated the inclusion of a table which explicitly summarised the hypotheses, research questions and corresponding results. The experimental results supported both hypotheses. We found the results very interesting, and the paper sparked off ideas for future work in the area. Judith and Jim discussed the possibility of conducting a similar experiment with children who have autistic spectrum conditions, an idea which Ellie had raised when drawing the paper to people’s attention. Given that the study focusses on perceived agency there are many questions around theory of mind and social interaction which could be addressed in future work.

There were a number of questions which occurred to members of the group after reading the paper. We were unsure whether the competitive and cooperative tasks were well-matched, and thought it might have been a better comparison if an action-based collaborative task was used. Additionally, the short time period of 2 minutes raised questions about whether differences would level-off over a slightly longer period. We weren’t sure whether the trading activity was likely to bring examples of truly collaborative behaviour, particularly given the short-term nature of the task and resulting lack of wider significance of outcomes. We would also have liked to know more some of the self-assessment measures, such as the way the Self-Assessment Manikin was used, and how co-player likeability was measured. So, we came away with unresolved questions, but we all agreed that it was a highly interesting paper, and that there was much merit in the work.

Edgar attended the VL/HCC conference in Innsbruck last year. This was his report to the lab group on the event.

Last October I went to the VL/HCC conference for the first time. I was part of the Graduate Consortium (GC) presenting the main elements of my PhD research in 10 minutes. The GC was the first activity of the conference (on Sunday). I have to say it was a really satisfactory experience – I received positive feedback and got to meet some young researchers from the USA working on similar topics. The main GC’s topics were Programming, Software Engineering. There were some bits of Security, SpreadSheets and Modeling for Engineering processes.

Some GC participants

Fig. 1. Jill Cao from University of Oregon (Left), Michael Lee from University of Washington (center) and me :).

Since there were considerable amount of papers, I’ll spotlight just a few that may be interesting to explore in detail, probably with some bias on the programming topic (sorry about that :)). For more detail, please try the conference webpage or the IEEE digital library.

From Monday to Wednesday there were 8 sessions: SpreadSheets and Tables, Design and Notation, End Users: Mobile Devices and Programming, Symbols and Notations, Code Understanding, Domain-Specific Languages, New Approaches to Program Specification and Social Computing.

I was a bit surprised about the spreadsheets topic; it seems this track has been “regular” during previous conferences. I don’t have any real knowledge about this field, so I’m afraid it all looked like “black magic” for me.

The first paper of the second session, “The Shape of Empty Space: Human-Centred Cognitive Foundations in Computing for Spatial Design” by Mehul Bhatt, Carl Schultz and Minqian Huang presented a model and framework to help architects providing “human-centred models” of special designs rather “engineering models”. My understanding without architectural knowledge is that this model will allow architects to analyze other views of the space designs, which may improve the “user experience”. It may be defined as “automatic building usability tests”. They showed a CAD application based on the model that can assist during the stages of spaces design.

During the third session (End Users), “An Exploratory Study of Blind Software Developers”, by Sean Mealin and Emerson Murphy-Hill (North Carolina State University) was presented. Sean is a blind student at NCSU. They presented the result of interviews with blind software developers focusing what challenges blind people must face and when writing code. The presentation finished with a quick demo tool for screen reading and coding on the Eclipse interface. It will be interesting to follow further developments and research.

The fifth session (Code Understanding) started with the paper “GUI-Driven Code Tracing” by André L. Santos. This is the best paper VL/HCC 2012. Andre presented an interesting prototype that allows GUI debugging. The prototype is based on the idea of “GUI-driven code tracing— to enable developers to navigate from UI elements of a running program to related source code”. Initial results showed that the debugging prototype could be useful when complex GUI code needs to be maintained.

The sixth session included the paper “Blocks Languages for Creating Tangible Artifacts” by Franklyn Turbak, Smaranda Sandu, Olivia Kotsopoulos, Emily Erdman, Erin Davis and Karishma Chadha. They developed two visual blocks-based TurtleBlocks and PictureBlocks, which produce tangible objects as output using laser cutters and vinyl cutters (Fig. 2). The paper discusses some interesting visual language design aspects, regarding with concepts such as connector shapes, typing, and polymorphism. The feedback collected from a workshop indicate that students “were more motivated to create designs when they could get tangible output from the turtle and picture worlds as opposed to just drawings on the screen”.

Fig. 2. Some tangible “output” where presented to the conference audience.

Fig. 2. Some tangible “output” where presented to the conference audience.

The seventh session (New Approaches to Program Specification) included the paper “Investigating the Role of Purposeful Goals on Novices’ Engagement in a Programming Game” by Michael J. Lee and Andrew J. Ko (University of Washington). I’ve read some work from Michael Lee, since he’s interested in novice programmers. Michael developed an online game called “Gidget” which is used to teach basic design algorithms aspects using an imperative language. This paper presented the results of an experiment where three different set of characters were tested as condition of engaging (Fig. 3).  They found that “1) those using vertebrate elements completed  twice  the  number  of  levels compared  to  those  using  inanimate  elements,  2)  those  using vertebrate  and  invertebrate  elements  spent  significantly  more time playing the game overall compared to those using inanimate elements, and 3) those using inanimate elements were more likely to  quit  the  game,  especially  on  difficult  levels. These findings suggest that the presentation of game elements that influence the purposefulness of goals can play a significant role in keeping self-guided learners engaged in learning tasks”

Gidget conditions

Fig. 3. The three conditions (game characters) tested in ‘Gidget’.

Last session (Social Computing) was focus on visualization of information. “KikuNavi: Real-time Pedestrian Navigation based on Social Networking Service and Collective Intelligence” by Hikaru Nagasaka, Makoto Okabe and Rikio Onai (The University of Electro-Communications, Tokyo, Japan) presented a real-time pedestrian navigation system as a social networking service operating on a mobile phone. It is a interesting concept to use the social network as a service. However, the paper is missing a formal user test.

Finally, my overall impression was highly positive. I found a friendly environment, which is really important for those who are “freshers” or less experienced presenters. The venue, Innsbruck, has really impressive views from the Alps, good food and tasty wine; all this helped to have a more pleasant conference. Looking forward for the next one.

View of facilities

Fig. 4. Outdoors view of the Congress Facilities