Monday, July 9, 2007

Progress: Testing Phase

It's been quite a long time since I updated on the progress of this project, but fortunately no news is good news in this case. The last two weeks of June saw lots of progress on all fronts, as well as my last summer meeting with my advisor. Everything is in order, and I'm ready to begin testing my three methods to determine the overall strongest method of interacting with the users of SL in creating music.

I've decided upon four criteria to judge my three approaches:

- Response Time
          Time between initial user input and aural response
          Measured in (ms); Lowest value is most efficient

- Accuracy
          Number of messages in compared with number of aural events out
          
Measured in whole integers; Lowest value is most accurate
- CPU Usage
          Percentage of processing power used when receiving user data
          Measured in a percentage from 0% to 200% on a dual core
          processor; Lowest value is most efficient

- User Friendliness
          Beta tester opinions on ease of use
          Measured on a 5-point Likert scale; Highest value is easiest to use


In addition to these criteria, I will also be testing in several areas of SL to take population strain on the server into account. Testing will begin next week after I return from the ProvFlux new media conference in Rhode Island. A more detailed post on the changes in the final coding for all the paths will be up soon.

- M

Tuesday, June 19, 2007

Progress: Wrapping up

The chat logging portion of this project is so close to being done, it hurts. Here is the data chain as it stands right now:

SL --> email --> POP3 PHP script --> OSC --> Max/MSP

Using this method, I have successfully gotten data into Max/MSP from SL in something close to real-time. The original test involved clicking a link to send the data; that worked fine. For the past few days, I have been working to incorporate the code that forwards the message through OSC into the POP3 script. So far, no luck on that. There are all sorts of possible reasons for this, the top suspect being 'REMOTE_ADDR'. I've tried multiple work-arounds for this issue, but still no luck on either the schools server or my local server. Of course, the problem may lie elsewhere, but I can't find it if so.

One other minor issue I'm having is with the ;max launchbrowser message. I was planning on hooking up a metro to the message in order to request data at regular intervals, but the message opens up a new browser tab every time the metro sends it a bang. I've searched google, the Cycling 74 forums, and all the Max documentation, but haven't found any messages that can refresh a webpage instead of opening a new one. I don't think this will be a big issue; it is simple enough to insert one line of script into the POP3 PHP file so that it will refresh once every X seconds.

Once these two small issues are fixed, this path is essentially done; I could then turn my test patch into a fully functional one and begin beta testing and small test performances. Hoping for a solution!

- M

Wednesday, June 13, 2007

Defining Second Life

A few days ago, a discussion popped up on the Second Life Educators mailing list about how to define Second Life in relation to other online virtual communities, specifically World of Warcraft. Though there are many different buzz words used to describe the various types of virtual communities, most agree that there is one specific difference between a world like WoW and one like SL: the presence of goals. Here are some quotes from the members of the list:

***
The basic distinction between SL and WoW for me is that SL is a simulation, and WoW is a game. Here are a couple of quotes from Akilli (2007):

"A simulation is defined as an interactive abstraction or simplification of some real life, or any attempt to imitate a real or imaginary environment or system (p. 4)"

"According to Prensky (2001) simulations and games differ in that, "simulations are not, in and of themselves games. In order to become games, they need additional structural elements--fun, play, rules, a goal, winning, competition, etc. (p.5)"

So while SL may contain games to play, it is not a game in and of itself. Akilli also uses the term game-like learning environments, "authentic or simulated places, where learning is fostered and supported especially by seamless integration of motivating game elements, such as challenge, curiosity, and fantasy. (p. 5)."

Finally, I'm noticing lots of folks using the term MUVE to refer to SL in literature, or immersive virtual world environment.
***

If you look at WoW and SL as virtual worlds, there is no difference.
Every MMORPG is also a virtual world. But not all virtual worlds are
MMORPGs.
***

I'd suggest that the common terms used are Social Worlds and Game Worlds/MMO/MMORPGs.
Social worlds are those where ludic elements are not coded into the system, they stem from tinyMUD, LambdaMOO etc, and go through to Second Life, There.com etc.

Game words are those where game elements such as character classes, levels etc are coded into the system, they stem from DikuMUD.

Yes people play games in social worlds and are social in game worlds, the distinction is more about the artifact and its intended purpose rather than necessarily about the emergent practices.
***

There was also a source cited by two people that I'm going to try to get hold of:

Akilli, G. (2007). Games and simulations: A new approach in education? In D. Gibson, C. Aldrich & M. Prensky (Eds), Games and simulations in online learning (pp. 1-20). Information Science Publishing: Hershey, PA.

Found it on Amazon, but hopefully I can find it in a library.

- M

Tuesday, June 12, 2007

Progress: New Path

For the past week, my advisor and I have been examining two new methods of getting data out of SL and into Max/MSP. The first still involves heavy PHP, but instead of using Silo to store the data it'll use a MySQL database. This solution seems much more elegant and stable than the first, but will involve quite a bit more PHP study.

The second method is already well under way. I have scripted an object within SL to send every chat message it hears as an e-mail. It is done and tested, so the next step is to automatically retrieve the e-mail and format the message so it can be sent through OpenSound Control. I'm currently hung up on that part, unfortunately. I've been following this tutorial on using PHP to check e-mail through POP3, but I keep getting the following error:

POP3 connect() - Error: Can't connect to Server. Error: 110 -- Connection timed out

I believe I have the mail server address correct because when I change it, I get a server not found error instead. I'm not sure if the issue is on my end, or with Gmail where the account is. I found the following post on the tutorial's website:


>>Connected to mail server POP3 login() - Error: No connection avalible
>>
>>I am getting this error?
>>
>>Any one can advise me?

>If you’re on a server without root access, you can’t use any port below 1024.


That's a different error than what I'm getting, but I can see myself having that problem on the school's servers. Gmail's configuration page says that port 995 is needed, which is obviously below 1024. The school has been having network issues over the weekend because of power outages all over town, but I've tried it on three separate days now and still no luck. I'm going to fiddle with it some more before my meeting with my advisor today, but I think this might end up being another IT issue. Crossing my fingers for no power outages today!

- M

Tuesday, June 5, 2007

Back to Square One

Unfortunately, the visit to my university's IT department was fruitless. If anything, the situation is MORE confused than it was before. No one was able to puzzle out the problem with getting Silo running on the school's server, so it looks like this entire path may be a bust. I've sent one last e-mail to the developer of Silo, but I'm reasonably sure the problem is with the apache configuration of the school's server.

My meeting with my advisor today was spent brainstorming some new possible paths for the chat logging part of the grant. It looks like there are two equally viable paths we could take: one more complicated, but somewhat quicker in transferring data, and one that is actually quite simple, but contains many possible slow down points. Before Thursday I'll be diving deeper into LSL in an attempt to get chat data formatted as a URL to be sent to a PHP email form. With any extra time, I'll also be looking into applescripting.

My video tracking patch is more bogged down than ever. It's so bad that I'm only getting one voice out of my poly~ object. I've decided that I'm going to use two separate Max patches for the video tracking portion of the project. I'll be splitting my current patch up into two parts, the Jitter part and the MSP part. One patch will run in Max itself while the other will be in Max Runtime, and the two patches can communicate through netsend and netreceive. I'm hoping that, as others have experienced, my dual-core processor will be more efficiently used by splitting it up in this way. Depending on how much time the LSL scripting takes me, though, these changes may not be implemented for a few more days.

- M

Friday, June 1, 2007

Progress: Additional Help

Some helpful e-mails have come through from various people, giving me a few more options to consider.

Correspondence with Zero Linden, creator of Silo, has proven fruitful. The first issue to overcome was debugging...Silo was not passing the tests set to it by the python program included with the download. The issue was actually in the formatting of the url, which is highly dependent on your apache server configuration. Silo has now passed all tests and has created the data folders that it needs to communicate with SL.

Zero Linden was also kind enough to provide me with a smaller, more efficient script to relay chat data to Silo from SL. Effectively, all it does is send everything it hears to Silo, with each message overwriting the last. That should be fine, since I want the data to be passed along to Max/MSP as soon as it hits Silo. Here's the new script:

default {
state_entry() { llListen(0, "", NULL_KEY, ""); }
listen(integer ch, string name, key id, string what) {
llHTTPRequest("http://some-url/chat", [ HTTP_METHOD, "PUT" ] , what);
}
}

Though, I'm still having issues getting everything working start to finish. Zero clarified for me in another e-mail that LSL scripts run on SL's side of the internet pond, not on my machine, so Silo MUST be installed on a remote server. This is where Stetson's IT department comes in.

Through several exchanges with IT, it's been determined that it'll be far easier if I just go meet with them and explain exactly what it is I'm doing. They claim that there is a problem with the Silo code and that they're getting several different errors. I have an appointment to meet with them today, so we'll see how it goes. Hopefully this will remove the last roadblock on this path!

- M

Thursday, May 31, 2007

Helpful Links

Since I've been doing so much online surfing for solutions to my server and CPU issues, I thought I should post some of the links that have been particularly helpful with some notes on the information.

+ Getting beyond 50% CPU usage in Jitter
This C74 thread gives some helpful solutions on getting Jitter to utilize multiple processor cores. I'm interested in the concept of running one patch in Max and the other in Max Runtime, so that the computer recognizes them as two separate processes and will distribute them how it will.

+ FPS Issues with Upgrade
This thread discusses the sudden drop in FPS when upgrading to Max 4.6 and Jitter 1.6. There is apparently an message that can be sent to Jitter to tell it how many cores and threads to use, as well as an option that defaults to "on" when it didn't used to.

+ Jitter Message Reference
A page listing all the messages that can be sent to the object Jitter to alter the behavior of Jitter as a whole. Good to know!

+ CheckUpDown Error Guide
CheckUpDown.com has a page for every HTTP error code that details common causes for the error as well as possible fixes. Useful for figuring out exactly what problem the web servers are having with Silo.

+ Enabling PHP on your Apache Server
Self-explanatory. Was very easy to follow.

+ Apache Webserving with Mac OS X
A 5-part guide with tons of information on using the Apache server that comes with OS X.

- M

Non-Progress

Well, it looks like things are at a standstill for the moment. Seems as though both the school's server and my advisor's web server have different issues with Silo that we can't solve ourselves. Zero Linden, the creator of Silo, has been a BIG help in getting us through most of our problems, but now it's all in the hands of the school's IT department. It looks like the problem with the school's server is that Silo requires access to the DELETE method to function properly, and that ability is likely disabled for very good reasons. Hopefully they'll be willing to let us poke through some of their heavy security so Silo can do what it needs to do.

My video tracking patch is still having bizzare CPU usage issues. Though the metro is driving most parts of the patch at 100ms, I'm hearing notes anywhere from once every 500ms to once per second. That is, of course, variable depending on how hard the CPU is working. Often times, I get an odd sort of galloping effect. The strange part is that this seems to happen no matter how much CPU is being used by the online community interface. This being a dual-core processor, I usually end up with Max/MSP using anywhere from 65-95%, depending on which interface I use. SL usually takes up no less than 95% (causing Max to use around 70%), while World of Warcraft uses only ~35% (causing Max to use ~85/90%). Notes seemed to come less often, if anything, running Warcraft, presumably because it is more populated and there are more people to track. One would think that with more CPU available to it, Max would take advantage and be able to process everything being asked of it. Rather, it seems to be restricting itself to just one core and scaling based on that. Perhaps the cv.jit objects don't support multithreading? I'm going to go ahead and build a polyphonic synthesizer to insert into the patch and see how the CPU behaves after that point. I suppose if that doesn't go well then I'll have to look into using two computers for this portion of the project.

- M

Thursday, May 24, 2007

Progress: Fixes and Issues

The good news is that Silo is now working! The bad news is that I had it right all along and the university was just blocking my apache server. Three hours of frustration for a simple fix, but at least it works now. I also read through both the code for the chat logger and the code needed to send data out of SL and into Silo, and it looks like the changes needed are pretty straightforward. I hope to have the chat logger working and sending data to Silo by the end of today. My advisor has been investigating the other end of the data chain, getting data from PHP to Max/MSP via OpenSound Control, and that has been going well. Data is flowing through the chain, though we are still figuring out how to format the messages so we get the result we want from Max. All in all, the chat logging trial has been going well, and I hope to have a working proof of concept by the end of next week.

The video tracking has been moving along, but I'm afraid I may have hit an impasse. Despite the fact that I'm doing this work on a 2.33GHz Intel Core 2 Duo processor, I'm almost maxed out on CPU power while running both Second Life and my video tracking patch. I know the major problems are jit.desktop and cv.jit.faces, but I'm not sure what to do with them to reduce the CPU strain. The video is split into four quadrants, and right now the face tracking and demo audio are only running on one of the four quadrants. If the CPU is already near maxed out, then it would probably kill my computer to have all four quadrants tracking faces with full audio effects resulting. I've already disabled the Use Onscreen option for the jit.pwindow objects, inserted some speedlim objects, and taken out a few redundant things; I'm not sure what else I can do. I'm going to go through the Cycling '74 forums to look for advice, but I'm really afraid that it's going to come down to needing two computers: one for tracking and one for audio. I'm very opposed to that option, since one of my main goals was to make this project as mobile as possible, but if that's what it takes then so be it. We'll see if it needs to come to that.

- M

Wednesday, May 23, 2007

Progress: Face Recognition

My motion tracking patch now recognizes faces. I've programmed it so that it feeds the coordinates of the face into cv.jit.track and plays notes based on the movement of the face. I also threw in a quick fix so that there wouldn't be silence when the face is lost, but I'll be taking it out and replacing it with something more elegant. Everything has been tested within SL and so far, so good.

Though I've had mostly success with this patch so far, I believe I'll be grinding to a halt before too much longer. I've been having CPU usage issues lately, and the patch doesn't even contain all the elements it needs to yet. I know my two main issues are jit.desktop and cv.jit.faces, which the help file states contains a very CPU-intensive algorithm. The patch certainly needs a lot of cleanup, both for CPU reasons and for aesthetic reasons, so I that will be one of my focuses for today. I'd also like to see if I can figure out how to incorporate the LSL script that allows communication with Silo into the chat logger, despite the fact that Silo still isn't up on the apache server. Between those two, I think I'll have plenty to do today.

- M

Tuesday, May 22, 2007

Progress: First Noise

The motion tracker makes sound! Right now it's little more than a crude noisemaker, but I do indeed have sound coming from motion. Obviously a step in the right direction; however, the motion tracking still needs to be refined. Today will mostly be focused on incorporating either cv.jit.faces or cv.jit.features2track. I'm hoping that one of these two objects will provide more reliable tracking within SL than cv.jit.track does. If all goes well for the next few hours (or if I get frustrated and need a break), I may work on the MSP side of the patch. The short-term goal is a simple polyphonic synthesizer, and I'll add more features as the project progresses.

Still no luck getting Silo installed on my apache server. Time to go forum diving again, it seems.

- M

Monday, May 21, 2007

Progress: Fun with PHP

Thursday the 17th proved to be a very frustrating day indeed. I decided to focus on the issue of getting data from Second Life into MaxMSP by way of my laptop's apache server. Enabling the server is baby simple; a checkbox in system preferences. Enabling PHP on the server was somewhat more complicated, but this Very Helpful Tutorial on editing your apache configuration files in Terminal worked out perfectly. Tested, it works, all is well.

Now, on to the real issue: Getting Silo up and running.

Silo was created by Zero Linden for storing and retrieving data using HTTP. Silo comes in the form of a PHP file that needs to be installed onto the apache server. This is where my problems have erupted. Where to install the file depends on your apache configuration. It seems to me (with my very limited knowledge) that the file should be placed under the Document Root, which for Mac OS X is /Library/WebServer/Documents. I tried testing it using the small python program that accompanies Silo, and the test came up with an error.

The other two most common apache configurations require that silo be placed either in the user's public-html directory or outside the document tree. From reading through my apache configuration, I don't believe either of these are it, though I admittedly have little to no experience in these matters. Of course, the Silo ReadMe does state that there are other, more complicated apache configurations that he did not list. If that ends up being the case, I'll need to track someone down who is knowledgeable about these things.

Needless to say, after three hours of trying to puzzle this out, I was frustrated beyond belief. Time to move on to something else!

I began constructing my motion tracking patch using the cv.jit objects. I ran a few tests, seeing how they would track movement in SL, and everything seems to be fine so far. Currently, the issue is incorporating cv.jit.faces into the mix to track individual people. It doesn't seems to output any sort of coordinates for the location of the face, but I may be able to use the output of the draw object used to outline the faces somehow. I'm still looking into it, and hopefully I'll have an answer by the end of today or tomorrow.

Onward!

- M

Wednesday, May 16, 2007

Progress: Video Tracking

Step One: What externals to use?

After several failed attempts at downloading and installing SoftVNS (corrupted files?), I finally gave up and began exploring other motion tracking options. My forum diving over at Cycling '74 turned up three more possible solutions: Electrotap's TapTools, Eric Singer's Cyclops, and Jean-Marc Pelletier's cv.jit library. Surface analysis of the three suggests that the cv.jit library will likely be my best bet, based on extra features and budget considerations (it's free).

After spending some time with the cv.jit library's well-designed help files, I've found both the objects I need for motion tracking and some extra goodies that should prove useful. For instance, cv.jit.faces tracks the presence of a human face in a video. I paired it up with jit.desktop to see if it would recognize animated faces in Second Life and, lo and behold, it does. This opens up all sorts of possibilities as far as giving individual users control over specific parameters or their own personal track. While there is no cv.jit object specifically for color tracking, the suckah object should serve well enough.

More fiddling is forthcoming.

Grant Proposal Abstract

I suppose the best introduction to my research is to read the abstract for the proposal I wrote back in January. Anyone interested in reading the entire proposal is welcome to e-mail me.

Abstract
  In recent years, interactive music has emerged as a way of involving those without musical training in music composition and performance. The goal of this project is to investigate a new medium for interactive music, the online graphical community. Using the programming environment Max/MSP/Jitter, I will build three small computer programs to convert text and video data from the users of the online community into music. This allows people from all over the world to create music together, with or without any prior musical training. The final product will be an interactive real-time performance blending the physical performance space with the virtual one, both visually and aurally.