If you’ve followed this blog for any length of time, you probably know that we are big fans of the arts around here. Lisa is a writer and designer, as well as a poet… Did you know that Toby was a bassist in a rock band and one time played the Terrace Ballroom in Salt Lake City? How about that Scott Preston is a professional drummer with an actual degree from a music conservatory? Ryan McClead moved to New York to expose Broadway to his talents as a composer/songwriter, pianist, and singer… Ayelette Robinson is a magnificent Ballroom Dancer… Jan Rivers is a photographer extraordinaire… Mark Gediman is… wait for it… wait for it… a professional DJ!! And, my family and I have our own rock band called “The Bendy Straws”!! In fact, now that I’ve actually written out all of these talents, it seems that we should form a band and hit the road!

We all love our day jobs in law firms because it gives us the ability to feed our love of the arts when we go home at night. I think that if we all had a wish, most of us would love spending more time on our arts projects and less time at work each day. However, we all like to eat, have a place to live, and we all like to travel, etc… so, we will continue to focus on our day jobs for now. But what about those folks out there that make a living through their arts? One of my favorite bands is trying a new way to make a living through their music, and is using a new approach to helping fund that work through a micro-financing website called Kickstarter.

Kickstarter is set up to help musicians, film makers, artist, technologists, designers, chefs, and publishers finance their work by appealing to the supporters of the arts to pledge a dollar amount to help produce the creative project. As the Kickstarter website says, “this is not about investment or lending. Project creators keep 100% ownership and control over their work.” So, this is not angel financing or having to sign over 51% control of your project to someone that has money (but probably no real love for your work.) In return, the artists give something back to their supporters. It could be a music download, signed pieces of previous art, clothing, or some other form of gratitude. Think of it as almost like having a garage sale to raise money to do your next project. In addition to getting something “physical” for your pledge, fans also get the “emotional” reward of being a part of the project. Quite honestly, most fans love the “emotional” reward more than than any physical item they may receive from the project.

The “fine print” of the Kickstarter projects is this:

  • Projects must have a dollar goal that must be met at a certain time
  • If the dollar goal is not met, then the project goes unfunded, and no money will be collected
  • Kickstarter gets 5% of any successful project, plus the money is collected through Amazon, who takes another 3%-5%. So, the artists get approximately 90% of the total money collected.
  • According to Kickstarter, project success rates are about 44%.
  • Since April 2009, Kickstarter has raised over $75 million for over 10,000 projects.

Let me give you an example of one of my favorite bands, and their Kickstarter project.

The Bunny Costume I wanted!!

The punk-pop band, The Dollyrots, is actually going to its fans to help finance their upcoming fourth album by using Kickstarter as their platform for raising money. This is a band that has been around for over 10 years, and has put out three great albums, and was actually picked up by Joan Jett’s label, Blackheart Records. They also tour the world like crazy. My guess is that they could probably get a studio to back their project, but would also fall under the control of that label and wouldn’t have the freedom to really create the type of music that their fans have come to love. So, why not let the fans be the money behind the project? In return, the fans get a couple of things for their money. First of all, the band is giving away things when you pledge a certain dollar amount. These items range from music downloads, to having you come to the studio and being part of the gang vocals and handclaps. My favorite (and I wished I had seen this sooner so I could have pledged the money) is that they are giving away their notorious bunny costume that they used in the “Because I’m Awesome” video. (That would have been… well, awesome to get!!)

So, how is the project going so far?? I’d say that The Dollyrots have hit a home run with this one. The initial $7,500.00 goal was met in the first day! Now, they have a goal of $12,000.00 (which the met in less than a week.) Looking at the numbers this morning, the average contribution is just shy of $70 per person. Not too shabby! In fact, the pledge success rate was so good, that they have been able to move up their studio time up and start getting the next album together next month today. I think that The Dollyrots lead singer, Kelly Ogden, says it best when she talks about how important it is to this band to have their fans finance their upcoming record.

“Let’s do this, guys. I promise this will be our best record, ever! … We believe in you, and we know you believe in us. That’s why this is going to work.”

In the age of flattening worlds, instant communications, and being able to find your niche, no matter how small or obscure, it seems to make sense that traditional funding methods aren’t the only way to get your projects financed. Projects like this, and platforms like Kickstarter prove that there is more than one way to get things done. This should have fans clapping, and studio executives nervous. I, for one, am looking forward to hearing this upcoming “fan funded” album. Hopefully, other artists out there can learn from this experience and use their fans to micro-finance their next project.

Two different NYT stories caused me to start thinking about crowdsourcing: World War II Mystery Solved in a Few Hours and Identifying Looters and Lovers in Vancouver’s Riot. In both instances, photos were posted online to help identify people who held themselves out in a public manner and whose actions were memorialized by photos. In a matter of hours, people began identifying those caught in the act. So it makes me wonder: perhaps George Orwell had it wrong. There’s no one Big Brother. There are Big Brothers and Sisters. B2S, if you will. Don’t know if I like that so much. Its bad enough being a Catholic and being guilted by the possibility of Big Eye in the Sky. It’s like I’ve said before—privacy is heading out the door …     Men are only as good as their technical development allows them to be. George Orwell

Somewhere between reading Jason Wilson’s post on “Exploded Data, the Legal Web and What We’re Missing” and Toby’s post on “KM 3.0 = Analysis“, my brain started to smolder from all the ‘future of data’ discussion. Jason takes an example of a 33 word sentence and how 66 individual pieces of “exploded data” (which looks a lot like XML structured data) were extrapolated, and the ‘explosion of data’ could probably have easily continued on for at least another 66 categories. Toby had talked about the predictions that the amount of information stored in the world today surpassed the zettabyte threshold and is continuing to grow as we get “better-faster-cheaper search and retrieval systems.” Wilson ends his post with these two sentences:

The question is whether we will step up to organize this sea of data, or wait until a program can do it for us. If the latter, what does it say about the future of legal research and the practice of law?

I don’t think that the problem is a “man vs. machine” issue, but rather a how can man make machine better, and vice-versa. Regardless of the amount of crowdsourcing you throw at legal information, there is just no possible way for humans, alone, to explode the information. Add to this the fact that there is no way for a computer algorithm (at least at this point in history) to do this either. Right now, it has to be a combination of efforts of humans working to ask the right questions, set the right conditions, and bring in the right experiences in order to help the computer algorithms or data structure establish methods of creating “smart information.” The overall process is for the human editors (curators) to get the information more quickly from “dumb” to “smart” through the use of technology, then work on tweaking the technology from time to time in order to keep make the smart information even smarter. Just as with many industries, the core functions of legal editing will become more and more commoditized, and the work will focus on establishing the next piece of smart information rather than focusing on uncovering the hidden data of existing information. Toby told me today at lunch that currently “humans ask the better questions when it comes to how to handle information.” He went on to say, however, that there will probably come a day when machines may actually start being better at asking the questions. What does that have to say about the future of information?

Hats off to Thomson Reuters for testing out the ‘crowd’ to help them build mobile apps for their Thomson Reuters API through their “StreetApps Challenge“.  There are six “prizes” for apps builders ranging from $15K in cash to an iPad for external contestants, to “Thomson Reuters Points” and “Shown on Thomson Reuters Jumbotron in NYC” for internal TR employees.  There is a long list of official rules, but here’s the two rules that I like the best to show what TR can get in return for its $25K investment:

  • All submissions must use the Thomson Reuters API, may employ other, external data sources, and must operate on a qualified mobile device for the Blackberry, Android, iPhone, and iPad platforms.
  • Thomson Reuters and its subsidiaries will retain all Intellectual Property rights to any applications chosen to receive prizes, and may use those applications as they wish in the future.
So, for $25K they could end up with six working mobile app for four platforms, and retain all IP rights on these apps (including any money they make off of them.) Brilliant!!  This particular competition is for TR’s financial products, but if this is successful maybe they’ll start expanding to their legal products as well. One of my friends commented to me that if it is really, really successful, they can start laying off more people.
It makes me wonder if Toby and I can pony up $25 (that’s twenty-five dollars, not thousand), if we could get one of our readers to develop a mobile app for the 3 Geeks blog.  Of course, that depends upon whether Toby has $25 in his wallet right now (’cause I’m a little short.)

As I mentioned yesterday, a group of bloggers traveled to ThomsonReuters (TR) in Eagan, Minnesota earlier this week to get a first-hand look at WestlawNext (WLN) and talk with the Project Cobalt team, meet briefly with TR’s CEO of Legal, Peter Warwick, and discuss the the functionality of WLN with Westlaw’s Reference Attorney staff. There are a number of articles that are out from other bloggers that a range of issues from Lisa Solomon’s discussion of Product & Pricing; Jason Eiseman’s video interview of myself me, Tom Boone and Jason Wilson; Robert Ambrogi’s discussion of West Search functionality; Betsy McKenzie’s view of WLN from an academic perspective; Ken Adam’s survey on CALR value in contract drafting; David Bilinsky’s Top 10 list about WLN, and; Simon Chester’s discussion of WLR from a Canadian perspective. I wanted to take a different approach and talk about the back-end structure of the new West Search Engine and how they have used Knowledge Management theories to create an algorithm that looks to be much better than the current Westlaw.com search results.
WestSearch – Leveraging 100+ Years of Knowledge
Westlaw has compiled millions of pieces of value-added information through editorial analysis of its research attorneys, but that information has been compartmentalized into individual databases and has had a slow transition from the traditional print uses of this information to the computer database search engines. Even when West created KeyCite as a competitor to Shepard’s, it was pretty much still a stand-alone citation system that added-value to the individual cases and statutes, but not really a great enhancement to how searchers retrieved results from their searches.
Now WLN has finally seen the value of four distinct products/processes that not only help as individual value-added products, but actually determine search results rankings, pushing better results to the top based on past knowledge rather than by simple algorithms of term location or dates. Here are the four products/processes that now affect the ranking of search results:
1. West’s Key Number System
2. KeyCite Citation System
3. West’s Secondary Catalog
4. User Searches & Resulting Actions
The first three categories are really no brainers when it comes to leveraging related pieces of ‘information’ on a similar topic. Now, instead of just getting results that have a term frequency that matches the words in my search query, WLN literally runs an algorithm of over 60 queries that determine alternative words that may apply to my search, top cited resources on that topic, and resources that are deemed to be authoritative through secondary resources. I like the fact that the Project Cobalt team stuck to the term “algorithm” and shied away from “artificial intelligence” (although Peter Warwick did let the AI term slip in his talk.)
Perhaps the most intriguing part of the change in algorithm of WestSearch is number four – “User Searches & Resulting Actions.” This is something that those of us interested in improving Knowledge Management resources see as the holy grail of KM. If we could create a system that leveraged the experience of everyone in our firm in a way that allows our KM tool to become “smarter” we’d jump at that chance. From what I saw, it looks as if WestSearch is making a strong run at making their search algorithm smarter through adapting results based on previous users key actions. In a way this is crowdsourcing on a very high level.
Crowdsourcing the Researchers
I’m a fan of crowdsourcing, and of knowledge management. Combining the two is like putting peanut butter and chocolate together and coming up with a two great things that work well together. WestSearch crowdsources the WLN users by monitoring key actions after the search results are returned. WestSearch takes actions like ‘print’, ‘save’, ‘folder’, and ‘view’ that a searcher performs and logs that information for future reference. If another researcher runs a similar search later, the actions from the previous users is taken into account and the results are influenced by those previous user actions to potentially rank items higher or lower in the results list. The thoughts behind leveraging previous user actions against search results is that the ‘crowd’ will tend to identify and use the same documents when searching specific legal issues. When pressed on this, the Cobalt team said that actions like ‘print all’ or ‘save all’ are not logged because they are looking for specificity over generality. Also, because it is a ‘crowd’ based view, the law student that doesn’t understand the issue and picks less valuable documents will have little influence because the majority of the crowd will choose the ‘best’ documents over and over again, thus improving the algorithm.
What’s the Future?
It will be interesting to watch as this rolls out and WLN adds more and more databases to the system. I will be really interested to see what happens when the “news” portion is integrated and how that will affect the WestSearch algorithm on ‘new’ or ‘hot’ topics that pop up from time to time. I look forward to watching this approach of leveraging existing knowledge in an advanced algorithm to see if it does get ‘smarter’ over time. I’ll also be interested in seeing if WLN can improve its existing West km product (which will still work with WLN), and the ‘foldering’ features that could actually make WLN an additional in-house KM resource.

Hats off to the folks over at Law Shucks for coming up with a brilliant, yet very simple way of tracking those lateral partner moves between big law firms.

With the Lateral Tracker, people within the big law firms who are “in the know” when attorneys jump ship from one firm to another have a chance to update the Lateral Tracker in order to let everyone know. In reward, the person submitting the information gets a chance to win an iPhone 3GS (and, probably a 2-year commitment to AT&T… so, it’s a win-lose deal.) This is the sort of crowdsourcing exercise that is perfect for this type of information.
In the old days before this Lateral Tracker, we used to have to wait until a press release went out announcing that Lindsey McDonald, former Partner with Wolfram & Hart, has now joined the Denny Crane Law Firm in Boston. With tips from people within the law firms, we can now know that McDonald is leaving one firm and joining another before he actually makes his trip from Los Angeles to Boston. In an environment when “first to blog” is the new ‘breaking news’, this type of tracker can be a great resource in tracking movement between firms.
There are probably lots of other crowdsourcing information that could be gathered on big law firms. For example, how about setting something up to list the names and contact information for all of the Marketing Department personnel within big firms? Most firms do not list the flunkies administrative people in their directories, but sometimes you really want to know who is doing the business development, or marketing, or knowledge management, or research work in a firm but don’t want to have to try to Google that information and hope that you find it.
I’m very impressed that Law Shucks has made an effort to create an ongoing crowdsourcing tool and hope that it turns out to be a great resource for all of us.

On my way in to work this morning, I overheard someone say “all these new ‘hot’ areas of law that weren’t even around 10 years ago.’ It was an existential moment that got me wondering what they were talking about, and then to what are the ‘hot’ areas of law that weren’t even around ten years ago. So I did what I usually do with these type of issues, and crowd-sourced it out to the Twitterverse to see if anyone came up with suggestions of what is a ‘hot’ area of the law today that didn’t exist in January of 2000. I thought I’d start the list off by suggesting a few things off the top of my head:

  • Green Energy Practice
  • Sub-Prime & Financial Crisis Practice
  • Electronic Discovery Practice
  • Guantanamo Bay Practice
Well… no good ‘off-the-top-of-my-head’ discussion goes without someone pointing out that I am wrong…. Immediately some smarty-pants Dallas lawyer and E-Discovery expert (who happens to know a lot more about this than I do) points out that E-Discovery was a 20th Century invention, and even points out that Texas ruled on ESI way back in 1999. Although, we eventually agreed that E-Discovery became “hot” in the last 10 years. My favorite comment was that although E-Discovery became ‘hot’ in the last 10 years, here’s wishing that it wouldn’t exist in 10 more years!
My co-blogger Lisa came up with a crazy thought that ‘virtual law‘ needs to be created and become a ‘hot’ practice area, but that ‘virtually’ no one is listening to her… ‘literally’.
I thought that maybe “Gay Marriage” issues will become a ‘hot’ area, especially when it comes divorce time. And, it was suggested that ‘e-commerce‘ law has also become a ‘hot’ legal practice area in the past 10 years (although it was also a 20th century invention along with e-discovery.)
What have I missed? Any other ‘hot’ legal practice areas that have sprung up in the past decade? (Yes, you have now become my ‘crowd-sourcing’ experiment!)

I’ve seen a lot of comments on “what is next for Web 2.0” in the past few months, but most of those jump ahead to the “Web 3.0” stage of Cyberdyne’s Skynet artificial intelligence stage. I wanted to bring the discussion back to a less scary (well, at least not “end of the world” scary) topic of the next stage in Web 2.0, which I’ll call, “Web 2.1 – The Unsolicited Interloper.”

Web 2.0’s Controlled Discussion

The 2.0 stage moved us from a one-way online discussion into a method of two-way or multi-way discussion. Blogs allow us to comment; newspapers allow people to submit comments on just about any story they publish; and social media tools allow us to chat and connect with peers in a real-time environment. One of the things with the 2.0 world is that whoever writes, or manages the content usually has some type of control over the discussion that follows within the four walls of that content-even if the only recourse is to delete the discussion, or moderate the comments before allowing others to see them. The key to 2.0 is that you can have four walls around your content. However, the 2.1 stage could tear down those walls and allow interlopers in, and you’ll have very little control over what they have to say.
Two Web 2.1 tools – Google’s Sidewiki and DotSpots’ “Distributed Objects of Thought”
Two of the leading contenders to usher in my version of Web 2.1 are Google Sidewiki (see video demo) and DotSpots (see video demo). Both of these are browser extensions that allow anyone to place unmoderated comments on any webpage and allows anyone with the same browser extensions to view those comments and add their own comments as well. Sidewiki does as its name suggests and puts comments into a sidebar, while DotSpot places the comment ‘dot’ right in the text of the blog. Now, neither of these actually does anything to the blog post itself… it is simply a feature of the browser extension that allows the modification on the side (sidewiki), or within the text of the browser page (DotSpot). If you haven’t added the extension to your browser, then you wouldn’t see the comments at all.
Free-flowing commentary versus Free-Flowing SPAM
The idea behind the Web 2.1 tools is noble. Using the wisdom of crowds, information can become free-flowing, and more informative by allowing those reading it to also add information. The DotSpot video is a prime example of what the developers of the product wish to do. Now an article can have comments pointing to additional information and the ‘crowd’ can begin interacting with each other and add additional content (maps, videos, etc.) making (or rather morphing) the original information into something dynamic. No longer is the ‘crowd’ limited by the restraints of the website’s comments section. It is this freedom that is worrisome for most of us that develop original content on the web.
In the Web 2.0 world, I can delete comments that appear on my blog. In this new 2.1 world, I cannot. Instead, according to what I’ve read in the FAQ’s for the two resources, I’d have to request that content be removed because it is SPAM or abusive in some manner. This is the part that I think most of us would find most troubling. It is one thing for another blogger to rip my posts to shreds on his or her own blog, but to essentially add any comments you want and have it show up through the extensions on my blog seems to be something that I’d rather not have to deal with. Yes, I know that it isn’t “really” on my blog… it is on Google’s Sidewiki or DotSpots databases (somewhere in the cloud), but if you have these extensions installed on your browser, it can sure look like it is.
The Benefit of the Unregulated Crowd
Maybe the inconvenience of the occasional spammer is outweighed by Web 2.1’s dynamic platform. We’ve grown accustomed to being the moderators of the conversations over our writings. Losing some of that control is not a comfortable thought for me. But, maybe it is not a bad thing to loosen some of that control. Most things I’ve read on the topic of crowds say that the ‘unregulated crowd’ creates a better result over one that is. Perhaps I should just take the good with the bad when it comes to the unsolicited interlopers that the Web 2.1 world brings.

It seems that Toby and I have a few extra dollars in our pockets, and we wanted to test some more Crowdsourcing Projects. Back in May, we did a 5 Part Series on Crowdsourcing and had some fun testing out different projects using Amazon’s MTurk Crowdsourcing service. We thought we’d test the value of Google Scholar Legal Opinions & Journal (SLOJ) using a Crowdsourcing Project . This time we’re not only asking our MTurk workers to do most of the heavy lifting, but that we’d also like to outsource the topic of the project to our blog readers. We are the ultimate Delegators!

If you have a project that you think would make a great add-in legal research Crowdsourcing project, put it in the comments below. Remember that the best Crowdsourcing projects are those that ask the workers to perform specific tasks that result in specific results. For example, let’s say I wanted to pull a list of URL’s from the new Google Scholar Legal Opinions & Journals site that match the cases in a volume of a law reporter. The project would first identify all of the cases within that reporter, then submit that list to the worker with instructions to search Google SLOJ, identify the specific case, then cut and paste the URL into the appropriate answer box. This type of one question, one result project works very well with crowdsourcing.
The type of projects that don’t work with crowdsourcing is the one question with many answers project. For example, if you asked the workers to search Google SLOJ to find every 2009 case in New York that deals with Eminent Domain issues, that doesn’t work very well because you’ll either have to assign one person to do all the work, or assign multiple people to do the same work over and over again. To make this type of project work, you’d first need to identify each of the cases dealing with Eminent Domain, then ask the worker to do something specific for each one of these cases. For instance, you could have them read the case then summarize the court’s decision. Or, you could have them identify specific information within the decisions such as who the attorneys are, who they represent, and which party the court ruled.
Now that you see the guidelines for submitting a Crowdsourcing project, let us know what you’d think would make a great Crowdsourcing project for Google SLOJ. Toby and I are ponying up the money for this project, so it won’t cost you a thing. We’ll compile and post the results right here on 3 Geeks.
Continuing our Crowdsourcing post from yesterday, here are the answers we got back for the final six:
Firm Name: Jackson Walker
Article Title: OIG gets enhanced funding for increased enforcement in health care
  • The Economic Stimulus bill gave the Office of the Inspector General nearly 30 billion dollars simply for increasing oversight over health service and care providers.
  • The stimulus plan also gives Inspector Generals the right to review any contracts or grants given through stimulus money.
  • The Inspector General Reform Act of 2008 is another piece of legislature aimed at increasing the independent authority of the inspector general and increasing the efficiency of the OIG in oversight over healthcare organizations.
  • OIG data shows that for every $1 the govt provides in funding, it recovers $17. The amount recovered in 2009 is already greater than the amount recovered in 2008. This means that we must expect increased OIG funding and oversight.
Firm Name: Kilpatrick Stockton LLP
Article Title: What must your healthcare organization do (if anything) to protect against patient identity theft?
  • Healthcare organizations must be in compliance with the Red Flag Rules of the Federal Trade Comission and are to create a Identity Theft Protection Program
  • Mandatory compliance with the Red Flag Rules is due in part because Healthcare organizations are both creditors and they have charged accounts, the two requirements under teh FTC to be under the Red Flag Rules.
  • By August 1, 2009, Healthcare organizations must have a Identity Theft Program in place in order to deal with issues such as payment transactions, consumer reports security as well instituional procedures to identify and reduce the chances of identity theft.
  • This Identity theft program will require the approval and involvment of the board of directors, will require that workers in the organization be trained in aspects of identity theft, and have transparent relations with outside service providers.
Firm Name: Dorsey & Whitney
Article Title: Take precautions now to prepare for Influenza A type H1N1 (formerly `swine flu`)
  • With the threat of a disease such as Influenza A type H1N1, businesses should have a plan in place in case it affects its employees.
  • A specific emergency plan should be put in place in case the disease strikes, focusing on such issues as chain-of-command.
  • It is important to have open communication with employees via non-traditional means to track any cases of the disease that are reported.
  • Attendance policies and sick-day procedures should also be reviewed so that employees who might be sick have options.
  • Perhaps the most important way to prevent the spread of Influenza A type H1N1 is through education; everyone should be well informed.
Firm Name: Finnegan Henderson
Article Title: Federal Circuit affirms award of attorneys` fees for litigation misconduct
  • Case is a medical device patent-infringement suit (spikes) between two medical supply companies, ICU and Alaris
  • ICU has repeatedly and variously claimed infringement in the use of specific spikes by Alaris, each time being rejected
  • Court found that ICU failed to disclose and specify between tubes and spikes in their cases
  • Court ruled against ICU and awarded attorney’s fees for Alaris for those portions related to spike claims
  • Under The Supreme Court 9th Circuit precedent, the awards held up during a final appeal
  • [comment from MTurker] This was a very challenging one—I spent a good deal of time on it, and did my best. I hope it’s good enough!
Firm Name: Fulbright & Jaworski
Article Title: FTC Delays Enforcement of Red Flags Rule Until August 1, 2009
  • Cutting Medicare spending will take a lot of “new offices and positions.” I’ll bet his “Office of Spending Oversight” will need 500 new expensive “experts.”
  • Increasing a budget by $1.7 billion to find Medicare and Medicaid fraud abuse is abuse to the American public.
  • Allocating $311 billion to physicians over the next 10 years will not cut cost of services. Doctors are not going to make less money, so services will be cut.
  • Making subcontrators liable for fraud will not work. Once care is given good or bad, it is almost impossible to track who is responsible for what.
  • Work plans and every other Medicaid fraud prevention plant will only add more expense to the already over inflated budget.
Firm Name: King & Spalding
Article Title: Obama Budget Proposal Includes $309 Billion in Medicare Medicaid Spending Cuts; $1.7 Billion Increase for Fraud Control
  • The US 2010 fiscal budget will increase spending in health and human services by more than 7% upto 879 billion dollars.
  • There will be budget cuts in medicare and medicaid programs by 309 billion dollars in order to save for the healthcare reserve fund requirement of 634 billions dollars.
  • A large proporion of the money going into this fund will be coming from competitive medicare bidding between hospitals and healthcare providers.
  • The govt is also increasing the money it spents on identifying and preventing healthcare fraud, focusing 1.7 billion over the next five years in order to save 2.5 billion in fraud losses.
It tooks us from Saturday morning until Tuesday to get 10 articles reviewed. We’ll probably need to re-run the test at a later time to see if it was the “weekend” or the “price” that caused such a slow turn around on the project. My initial feelings are that it is probably a combination of the two.
This sort of task requires the MTurker to put some thought into the process. I really didn’t see a huge drop off in quality between the results that we got back at 25¢ versus the 50¢ results. But, there did seem to be an increase in the time it took to get the answers back. So, if you have more time, you can pay “less”, if you need something done in a hurry, then you need to pay “more”.
I have to admit that I was pretty impressed with the quality of the work. Regardless of if we paid 25 or 50 cents, the work was very good. I’m also stunned by the seriousness that the MTurkers seem to take with regards to the quality of the work. Take a look at the last bullet-point of the Finnegan Henderson article. A MTurker posted a comment saying that they had some difficulty with the article but hoped that their results were “good enough” for us. That really impressed me.
The more I test the MTurk idea, the more I see potential in crowdsourcing a number of projects that we’d love to do within the law firm setting, but generally don’t have the staffing to help us complete the projects. We’ll break down some of the other MTurk projects we tested over the past week and show you what we’ve found to be the pro’s and con’s of crowdsourcing.