Late last week I learned that the first NeuroGaming Conference will be held on May 1-2, 2013 in San Francisco. The panel discussions planned for those days look like they would be very informative. They may even help me move forward with my little prototype a bit more quickly. Alas, if only I had known about it months earlier, when the discounted conference and hotel rates were in effect. San Francisco is an expensive place to visit from the east coast!
Maybe next year…
According to the Mindwave documentation, I have listed the ranges of eSense headset values and my interpretation of these ranges for attention and meditation:
In order to allow the player to play the level, I think it is best to start with lower thresholds for both attention and meditation. As the player learns the basic rules and movements for the game, then these thresholds should be increased to encourage / challenge the player. An example of a possible threshold cut-off for the prototype and subsequent game levels is shown below:
eSense(tm) Meters. (n.d.). Retrieved from Neurosky Developer: http://developer.neurosky.com/docs/doku.php?id=esenses_tm
My prototype does not use the meditation index value but I am beginning to reconsider its value. In a general context, meditation or the act of reflection or contemplation is an effort to self-regulate the mind or to generate an emotional state in order to analyze it. Being able to induce a particular state of mind, such as calmness, has been found to have a predictable effect on the mind and body.
We know that stress is a key disruptor of control on attention. Kaszniak discusses the cognitive impact on attention by emotional states such as stress. He describes some interesting findings regarding the benefit of a calm, focused emotional state (achieved via some form of meditiation) on improving attention.
While Kaszniak also states that these “claims of effectiveness [of meditative practices for children in educational settings] are premature”, I am now considering using the meditation index values along with the attention values in some way to control the character’s motion in my prototype.
Kaszniak, A. W. (2009). The Neuroscience of Attention, Emotion & Meditation: Implications for Education. Retrieved from Association for Mindfulness in Education: http://www.mindfuleducation.org/AttnEmotMeditatetalk.pdf
Lutz, A., Dunne, J., & Davidson, R. (2007). Meditation and the Neuroscience of Consciousness: an introduction. In P. Zelazo, M. Moscovitch, & E. Thompson, Cambridge Handbook of Consciousness.
As I finish work on my prototype, I have been reconsidering my understanding of attention and what it means in this context. Attention is generally understood as “taking notice of something” or “applying the mind to something”. From the perspective of cognitive psychology, attention is the “cognitive process of selectively concentrating on one aspect of the environment while ignoring other things” (Wikipedia). Current areas of research include understanding the relationship between other cognitive processes particularly working memory and sustained attention. A new area of focus is on how traumatic brain injury effects of attention.
Attention can be categorized:
By sense and differentiation—visual, auditory, etc. and whether it is directed toward a sensory source (overt) or focused on one of several stimuli (covert).
As Selective – this involves focusing (general and specific) a sense (i.e. visual attention)
As Divided – this involves actively paying attention to more than one task at a time
By Processing Type
Top-down, goal-driven or executive attention–This type is mediated primarily in the frontal cortex.
Bottom-up or stimulus-driven—this type is involuntary and driven by the property of objects such as motion or noise.
About my failures…
I have Unity reading data from the Mindwave device through the ThinkGear Connector using .NET socket protocol. I had this working a few weeks ago. I got overconfident and transferred my focus to learning how to build menus, learning about character controller scripts, and even investigated path-finding tools (available in Unity Pro). When I started to put this all together (rather quickly I might add) to have a demo for the Harrisburg Game Developer’s meetup last weekend, I discovered that :
- I started way too many mini-projects: two scenes, one with a heads-up display; a main menu that looked crappy; a progress bar script that wasn’t working correctly; and a few more concepts I didn’t want to lose.
- I didn’t have a good way to allow the user to test their headset connection before entering the game level (thus the additional scene)
- I had never fully tested the ThinkGear client (TGC) script other than viewing the log data in the console window. When I tried to read the data from other components, this is what I experienced:
Unity Debug Example (QuickTime)
Lag. Significant lag. The modified version of my earlier TGC script runs in Unity and is called within the Update event to reading and parsing the data. In OnGUI event, an attention value is supposed to be displayed. But I am not getting anything as Unity locks up, hanging even when I try to exit the Play Mode of the Game view!
After some thought and research, I have a few options to investigate:
- Pull the TGC script out of Unity and create a C# DLL (plug-in) that performs the same tasks but outside Unity.
- Create a co-routine that runs in the background but still within Unity.
- Create a thread within Unity. I am still learning what the differences are between these last two. Unity can have only one main thread so the additional threads cannot call Unity API methods. I think that my TGC class is almost set up this way.
I read a few very interesting quotes today by speakers for the upcoming IA Summit on failure and problem solving which really resonated with me (wish I could go!):
On failure and user experience, David Farkas says:
…is a rallying cry for our profession to be more comfortable with admitting, discussing, and sharing failed projects, code, discoveries and experiences with each other and our clients.
And on documenting failed experiments, he says:
..identifying the gap in knowledge in our field isn’t solely in academic or mentorship programs but exists in our fear of failure, or more importantly our resistance to openly discuss it.
Regarding my work on this prototype, I totally agree with Andrew Hinton’s comment on “making new stuff”:
Generally, I still get a sense that practitioners are impatient with complexity, and want to make everything too simple and concrete, too quickly. The zeitgeist right now is all about “making stuff” without much thinking, analysis or modeling. The assumption seems to be that if you’re working through a problem with any level of abstraction – analyzing, modeling – then you’re not making anything yet. This is horribly wrong-headed, and leads to a sort of blindness to big, complex, systemic challenges, and an over-focus on problems that can be tackled with a quick prototype. Meanwhile, the world is getting even more systemically complicated. We need to embrace complexity, then tame it.
This helps me remember that my little prototype is just that, a prototype, a means to get people thinking along a different line, out of the box. I am not a neuroscientist but, hopefully, I will speak to a few about this work and its potential as a therapeutic tool for attention and focusing problems particularly for those dealing with the long term affects of concussion-related injuries.
I guess the theme for today is failure or, rather, learning from failure. It’s a good thing. Unfortunately I have found that the work world (or rather the profit-focused corporate world) doesn’t believe this.
First a quick follow-up on my previous post about my bluetooth issue. I really can’t say for sure if my loss of bluetooth services was due to those Windows updates or not. Turning on the service from the services manager wasn’t an option as it was just GONE. I uninstalled the ASUS Bluetooth Suite, Mindwave Mobile (v.126.96.36.199) and Neurosky Mobile (v. 2.1.9) programs and rebooted. I hadn’t turned off my antivirus software the last time I installed these programs which may have been a mistake. I read that if this is not done then, while the Mindwave tools may appear to install correctly, they are not. So this time I was a little bit more anal in my approach:
- Turn off anti-virus (in my case, Avast)
- Install ASUS Bluetooth Suite – follow directions (this involves a few reboots)
- Install Mindwave Mobile v. 188.8.131.52 only. I did not install any of the MyndPlay software (v. 2.1.9) which appear to use an older version of some of the Mindwave components.
So far, so good.
I also learned that only one application at a time can access the ThinkGear Connector. This is a good segue into my next post on failure.
I had big plans for today. I had hoped to give a demo of my simple Unity application using the Mindwave headset to the Harrisburg Game Developer’s Meetup group. I needed to create a main menu and a simple level that would allow the user to test their headset connection before moving into the 3rd level. I worked all day to learn how to do this with Unity. By the time I had to leave for the meeting, I had bits and pieces done but nothing finished. But that’s okay. We are a group of learner’s so I knew that they could see past the unfinished state of my work. Windows took a bit longer to shut down because of some forced updates, fifteen of them. I didn’t think anything of it at the time.
When I booted my laptop up again, I discovered that something had dropped my Bluetooth service and scrambled my COM ports. Yeah! Why does this happen every time I want to make some progress?? I had hope to have pictures or a video of play testers (aka meetup members) using the headset playing one of the Neurosky applications. Oh well. Tomorrow will be spent trying to repair the damage. I just hope it doesn’t take all day. I have work to do!!
One a positive note, the group had a bunch of questions and expressed their interest in seeing my finished level at the next meeting.
- Neurosky Mindwave Mobile – The device comes with the MyndPlay applications, uses Bluetooth v2.1 and communicates via the ThinkGear Connector that is installed on your desktop. To connect to your iPhone or iPad, you just need to turn Bluetooth on. There is a free app available from the AppStore, Mindwave Tutorial App , that I find useful for testing the connection to and signal from the device.
- ASUS Mini Bluetooth Dongle – After I received my headset, I discovered that my laptop did not have Bluetooth so I ordered a dongle separately. This device has worked like a charm.
- Rechargeable AAA Batteries – The Mindwave uses a single AAA battery and works best when the battery is fully charged. Having a set of rechargeable batteries on hand will allow you to continue developing and playing. I noticed that my connection disconnects after several hours of use and the headset will not connect after several days of no use.
- Mindset Developer Tools – This is a set of software tools and APIs and is available for free from the Neurosky Developer Site.
- Unity – According to the developer wiki, Unity Pro is required in order to use the ThinkGear native library because the “library relies on Unity plugin functionality”. The wiki goes on to state that a “developer can work around these restrictions” but it requires installing daemon software. Being a casual Unity developer, I found this statement to be a bit confusing/misleading. My interpretation was that I HAD to use Unity Pro because the other method may be too complicated. So I ordered a student version of Unity Pro. What I now know is that it just means that if you have the free version of Unity then you can only read the device via ThinkGear Connector (TGC). This utility software is part of the standard desktop installation and is used by the MyndPlay applications (Flash applications) and the basic user tutorial. It loads at start up and sits in your system tray. I will discuss how to use it within Unity in a future post.
Time tracking – I wanted to use this project as an opportunity to find a flexible and inexpensive time tracking tool that I might use for freelance or consulting gigs. Harvest is that tool. I can create a custom list of project-specific tasks and enter the time I spend on from my iPhone, iPad and desktop which is very convenient.
Build a prototype immersive game or simulation using the Unity game engine and a non-standard user input device for casual use by patients for treatment of a stress- or pain-related health condition.
The Discovery (this is a synopsis of several weeks of discussions and background research)
For my applied project, I wanted to apply my software development knowledge towards building an immersive experience to assist in the treatment of a health-related condition. Initially I was planning on using the Microsoft Kinect as an input device for a stress reduction application developed using the Unity3D game engine. After discussions with my project mentor, Kel Smith, author of Digital Outcasts, we decided to focus on an application for pain distraction. The use of immersive technologies to create virtual experiences, such as SnowWorld (University of Washington), to assist in pain reduction or to provide pain distraction have proven to be highly effective. There is growing interest in using similar techniques for a broad range of health issues.
As I got further into my preliminary research of the topic of pain distraction, I began to consider that the Kinect may not be the best tool to use as the user input device for a pain distraction application. I then discovered some work using a brainwave headset to monitor patient attention levels. Click. Why not use a low-cost brainwave sensor like Emotiv or the Neurosky Mindset? A casual review of the Neurosky website revealed a developer’s area with information for “casual” developers like me on how to develop applications for various platforms–including Unity–using their developer tools. Perfect!
Kel loved the idea! We discussed how an application that monitors attention might assist in the treatment of concussion patients. I placed my order for the Neurosky Mindwave Mobile and started learning about mild traumatic brain injury (mTBI) treatments.
Thanks Kel for your guidance and continued support! Your knowledge and experience in the fields of healthcare and accessibility are greatly appreciated as I wander down this very new path.