|
|
Does Peer Review Really Work?
Last post 08-21-2009, 6:08 PM by 0ccam. 45 replies.
-
08-13-2009, 4:26 PM |
-
checkmark
-
-
-
Joined on 01-24-2009
-
Seattle, WA
-
Posts 59
-
-
|
Re: Does Peer Review Really Work?
Considering the frequency that it is pointed out there are some categories do not get reviewed fast enough if ever it would seem that email is not an adequate solution.
Every page on waymarking has the same indicators on the right side bar. For those that wish to cast their votes that seems enough notice. Adding an RSS feed is a much more comprehensive solution for many waymarking.com site updates. There are many RSS reader plug-ins for iGoogle and GMAIL.
|
|
-
08-13-2009, 6:29 PM |
-
the blue quasar
-
-

-
Joined on 11-04-2006
-
St. Catharines, ON
-
Posts 2,186
-
-
|
Re: Does Peer Review Really Work?
I think Peer Review works because of what we have for guidance in how to determine our vote. It reduces the subjective aspect and asks us to approach the Category not as something we would enjoy but if we feel that others would enjoy it.
And that is how I vote. My pariticipation in categories that I find lack-luster however, is unlikely. The icon does not mean enough to be to actively search out items that do not appeal to me but if I happen to find one then I may list one (if I have the time, which I rarely do)
BQ
|
|
-
08-13-2009, 7:03 PM |
-
silverquill
-
-
-
Joined on 11-11-2006
-
Cheonan, Korea
-
Posts 1,367
-
-
|
Re: Does Peer Review Really Work?
the blue quasar:I think Peer Review works because of what we have for guidance in how to determine our vote. It reduces the subjective aspect and asks us to approach the Category not as something we would enjoy but if we feel that others would enjoy it.
And that is how I vote. My pariticipation in categories that I find lack-luster however, is unlikely. The icon does not mean enough to be to actively search out items that do not appeal to me but if I happen to find one then I may list one (if I have the time, which I rarely do)
BQ
That's assuming that people actually read and apply the guidelines! What is the average number of people actually voting in peer review on any given vote? 75? Maybe in order to vote we should be required to check a box that says, "I have read the guidelines" like we do for creating a waymark. (When was the last time I actually read those???) There are guidelines given for creating a category, too, and I often wonder if they were ever read or understood.
Maybe there isn't a way to require people to post an idea to the forums before creating a category, but I think it might help if there were a prominent banner on the guideline page strongly suggesting that this be done before creating the category. Something on the order of a 24 pt flashing marquee comes to mind.
|
|
-
08-13-2009, 7:28 PM |
-
BruceS
-
-

-
Joined on 11-04-2006
-
St Peters, MO
-
Posts 3,569
-
-
|
Re: Does Peer Review Really Work?
silverquill:Maybe there isn't a way to require people to post an idea to the forums before creating a category, but I think it might help if there were a prominent banner on the guideline page strongly suggesting that this be done before creating the category. Something on the order of a 24 pt flashing marquee comes to mind.
One solution if they don't post to the forums is that the community start it as soon as it goes up for vote, as I have done in the past and just did again. This is less than desirable but at least discussion will be started.
|
|
-
08-14-2009, 3:12 AM |
-
Team Sieni
-
-
-
Joined on 01-19-2007
-
London, UK
-
Posts 435
-
-
|
Re: Does Peer Review Really Work?
checkmark:Considering the frequency that it is pointed out there are some categories do not get reviewed fast enough if ever it would seem that email is not an adequate solution. Every page on waymarking has the same indicators on the right side bar. For those that wish to cast their votes that seems enough notice. Adding an RSS feed is a much more comprehensive solution for many waymarking.com site updates. There are many RSS reader plug-ins for iGoogle and GMAIL.
I don't understand that - email is perfectly adequate for reviewing waymarks themselves and hundreds of those are dealt with for each new cat that comes up, so I can't see why it should not be adequate for category review.
I don't use RSS at the moment. It's possible that I could set something up alongside my email (I don't know how, but it may be possible) but as far as I can see from the web pages only New Waymarks and New Categories are available as RSS feeds (or have I got that wrong? - I know nothing about RSS)
|
|
-
08-14-2009, 4:23 AM |
-
the blue quasar
-
-

-
Joined on 11-04-2006
-
St. Catharines, ON
-
Posts 2,186
-
-
|
Re: Does Peer Review Really Work?
Team Sieni: checkmark:Considering the frequency that it is pointed out there are some categories do not get reviewed fast enough if ever it would seem that email is not an adequate solution. Every page on waymarking has the same indicators on the right side bar. For those that wish to cast their votes that seems enough notice. Adding an RSS feed is a much more comprehensive solution for many waymarking.com site updates. There are many RSS reader plug-ins for iGoogle and GMAIL.
I don't understand that - email is perfectly adequate for reviewing waymarks themselves and hundreds of those are dealt with for each new cat that comes up, so I can't see why it should not be adequate for category review.
I don't use RSS at the moment. It's possible that I could set something up alongside my email (I don't know how, but it may be possible) but as far as I can see from the web pages only New Waymarks and New Categories are available as RSS feeds (or have I got that wrong? - I know nothing about RSS)
I think that it meant that there are those that don't respond to the emails they are getting for new waymarks to review, so why would emailing about new category votes work any better?
As Waymarking has progressed, some aspects have outgrown the capabilities of the site itself, IMHO. I'd rather not get any emails from Waymarking.com, not even for visits. Instead I'd like an overview page of activity and things needing my attention. Recent Visit logs, Things to Vote on, etc.
On the topic at hand though, I'm not sure what the real concern is. Why is there a concern about what categories pass or fail Peer Review? I don't worry about what movies are available at my local Blockbuster as long as I can find one that I want to watch.
 BQ
|
|
-
08-14-2009, 5:03 AM |
-
Team Sieni
-
-
-
Joined on 01-19-2007
-
London, UK
-
Posts 435
-
-
|
Re: Does Peer Review Really Work?
the blue quasar:
On the topic at hand though, I'm not sure what the real concern is. Why is there a concern about what categories pass or fail Peer Review? I don't worry about what movies are available at my local Blockbuster as long as I can find one that I want to watch.
I must say I'm about half in agreement. However, I would get worried if my local Blockbuster was so full of awful home made movies I couldn't FIND what I wanted to watch, and if other people were put off by that and decided not to come to blocbuster at all for that reason. Ideally even the movies I don't want to watch would be well made and properly labelled. To put it another way, I don't really mind if say "key cutting stalls" or something I'm not interested in gets approved, so long as it's a good well written cat, but what bothers me if wooly overlapping cats for things I am interested in get approved. Or if so many cats for "key cutting stalls" "cheesegrater sharpening establishments" "places that sell exploding cigarettes", "cool places" get created the interesting stuff gets drowned out.
On the point about emails ... I understand now. I was just being slow on the uptake.
|
|
-
08-14-2009, 5:29 AM |
-
dinoprophet
-
-
-
Joined on 07-07-2008
-
Orion (OR-ee-un), MI
-
Posts 302
-
-
|
Re: Does Peer Review Really Work?
checkmark:Considering the frequency that it is pointed out there are some categories do not get reviewed fast enough if ever it would seem that email is not an adequate solution.
Every page on waymarking has the same indicators on the right side bar. For those that wish to cast their votes that seems enough notice. Adding an RSS feed is a much more comprehensive solution for many waymarking.com site updates. There are many RSS reader plug-ins for iGoogle and GMAIL.
I check email every day. I don't check this site every day, possibly for a week or more. Which brings to mind another idea: extending the voting period. Three days seems like forever when it's your category, but it might not be enough for everyone to realize there's a vote. Another idea: set the days that votes go up. When you submit a category, it goes up for vote on the next Saturday, say, with all the others that are ready. The vote runs until Wednesday or Thursday. People will then know when to check for new categories.
|
|
-
08-14-2009, 5:52 AM |
-
globetrotters.us
-
-

-
Joined on 09-15-2007
-
Ann Arbor, MI
-
Posts 877
-
-
|
Re: Does Peer Review Really Work?
dinoprophet:I check email every day. I don't check this site every day, possibly for a week or more.
Which brings to mind another idea: extending the voting period. Three days seems like forever when it's your category, but it might not be enough for everyone to realize there's a vote.
Another idea: set the days that votes go up. When you submit a category, it goes up for vote on the next Saturday, say, with all the others that are ready. The vote runs until Wednesday or Thursday. People will then know when to check for new categories.
I like that idea
|
|
-
08-16-2009, 4:19 AM |
-
Team Bubble Tea
-
-
-
Joined on 08-01-2009
-
-
Posts 3
-
-
|
Re: Does Peer Review Really Work?
HI we're newbies.
In Tasmania, where we are, there are several hundred geocaches (+500) within a 2 hour drive. There are 34 waymarks, and we've added 20 of those in the last month. (look it up, Tasmania, Australia, post code 7004).
We're not sure why waymarking hasn't taken off here. The north of the state doesn't use geocaching.com much, they prefer the Australian caching site after some "falling out" a few years ago. And we can see why. When we recently suggested a discussion about a new waymarking category we had two reasonable responses, and two responses that were not pleasant and didn't help us.
But back on topic (sort of), we love lots of categories. There's been some talk in this thread of "figurative statues". And of people wanting to reduce categories. Personally, I am absolutely sure there is no statue of an insect in Tasmania. BUT, I now have a mission, "to find an insect statue and waymark it". If these categories were reduced to a single "Figurative Statue" it would be too easy. Accepting that category reduction is the way to go would result in four categories of "Animal, Vegetable, Mineral, Metal" and we could all waymark these in a weekend then find something else to do.
Ultimately, category diversity is what makes waymarking fun.
Sure, there are issues of quality in waymarking. But consider that Geocaching Australia has essentially NO rules. You can place a cache right next to another if you want, it can be buried, stupid and dangerous. So if you want, you can do that. It's the freedom of the individual to be a great cacher (or waymarker) or a not very good one.
In the final analysis, if you set-up a poor waymark category or post poor waymarks, you're only cheating yourself. Peer review should be "Ok, they want to do this fine". And then publish tables of category performance...
<rant off>
|
|
-
08-17-2009, 8:44 AM |
-
chapterhouseinc
-
-
-
Joined on 11-05-2006
-
SWEPMT
-
Posts 1,795
-
-
|
Re: Does Peer Review Really Work?
i have long wanted a daily diges (i got those in my yahoo groups).....
some days i get no waymarking emails....but if someone goes to disney world, pinellas county, mammoth cave national park, and hundreds of other places my inbox fills up quickly--i often wonder when i am going to have to make a waymarking dedicated email address [if each of my marks was visited on the same day AOL would explode]......
and this digest would also eleminate some redundant emails (i forgot something, cancel the submition, and resub)....i know ive seen a few of these lately.....but then again, why exactly are we locked out of editing the description when subitting? well, at least not as many deny for capitolization errors at this point. shoot, many dont even notice/comment. like they dont even look at the submition anyway....
|
|
-
08-17-2009, 10:39 AM |
-
Redneck Parrotheads
-
-
-
Joined on 11-11-2008
-
North Port, FL, USA
-
Posts 538
-
-
|
Re: Does Peer Review Really Work?
I don't have any objection to notification emails for peer review, message digest (as long as they're optional), constructing a schedule for categories to go through peer review, or allowing peer review comments to be viewable (while still maintaining voter anonymity). I don't remember if that last idea was in this post or a similiar one but it might actually reduce the rudeness, as well as provide insight for other voters.
It'd also be lovely if peer review "nay" comments were kicked back if they did not provide actual words. "No" and "." would be unacceptable text entries for a "nay" comment. Heck, I'd love if "redundant" would also be included. I think a lot of people use that word without knowing what it means and if they're using it properly, they ought to be able to explain how a category is redundant.
Waymarking might really benefit with an article section that provides tutorials and tips from experienced waymarkers - like BruceS, silverquill or BQ (just to name three). The FAQ is informative but brief and often people do not read FAQs and other Help sections. People are more likely to read articles, especially when statements like "develop your category" are elaborated on how this can be done. Not everyone comes to the fora and information presented in an article would be accepted with greater credibility, anyhow.
It's likely Officer votes would also benefit from being more interactive. I've written categories where my officers were very involved in the process of determining instructions, constructing phrasing and correcting my typos. I've written categories where my officers couldn't be prodded with a stick and I actually stopped setting categories on a timetable (ie. "please let me know what you think by thursday...") because I wouldn't even get a "looks good" much less constructive criticism on the last day. Officer votes can be more interactive by allowing comments during officer vote and the category displayed with the voting field like it is in Peer Review. Plus an article on how a group should work as a team to develop the category.
I also think categories that are managed by groups that are populated by the same person with multiple accounts, should be suspended until proper officers manage the category. It's waymarking's version of shilling.
|
|
-
08-18-2009, 2:30 AM |
-
silverquill
-
-
-
Joined on 11-11-2006
-
Cheonan, Korea
-
Posts 1,367
-
-
|
Re: Does Peer Review Really Work?
Redneck Parrotheads:Waymarking might really benefit with an article section that provides tutorials and tips from experienced waymarkers
Just to let you know, I'm heading up a group to reactivate Hikenutty's "Waywardwamarking" site which has some good guides for beginners. They project was put on hold during my recent trip out of the country, but I hope to get this back on track. I still think having a prominently displayed recommendation, in the current official guidelines, to post new category ideas to the forums might help. Some still won't, of course. And, I think I'm to the point of requiring a premium to have a minimum number of posted waymarks in order to be eligible to create a management group and new category. That would eliminate a large number of the shallow or inappropriate categories that come up.
|
|
-
08-18-2009, 7:00 AM |
-
Redneck Parrotheads
-
-
-
Joined on 11-11-2008
-
North Port, FL, USA
-
Posts 538
-
-
|
Re: Does Peer Review Really Work?
silverquill: Redneck Parrotheads:
Waymarking might really benefit with an article section that provides tutorials and tips from experienced waymarkers
Just to let you know, I'm heading up a group to reactivate Hikenutty's "Waywardwamarking" site which has some good guides for beginners. They project was put on hold during my recent trip out of the country, but I hope to get this back on track.
I still think having a prominently displayed recommendation, in the current official guidelines, to post new category ideas to the forums might help. Some still won't, of course. And, I think I'm to the point of requiring a premium to have a minimum number of posted waymarks in order to be eligible to create a management group and new category. That would eliminate a large number of the shallow or inappropriate categories that come up.
Addressing your latter point: Anyone can post any required number of fast foods and then presume to present a category that's ill-fashioned and conceptually stunted. I don't think that would resolve the problem, although I do think it could help to establish a precedent that Waymarking requires some contribution.
Addressing your former point: Waywardwaymarking was awesome and it's unfortunate that it could not continue. If Waywardwaymaking was still active, I would be making the same suggestion. The weight of the credibility for the articles would come from being hosted on site.
|
|
-
08-18-2009, 7:49 AM |
-
silverquill
-
-
-
Joined on 11-11-2006
-
Cheonan, Korea
-
Posts 1,367
-
-
|
Re: Does Peer Review Really Work?
True, nothing is going to guarantee that good categories get created, but requiring some minimum participation and experience would at least mean that someone has invested some amount of time and effort in becoming familiar with how waymarking works, and make it more difficult for people to just flit in, create a cat, then flit out, not knowing what they're doing. Not foolproof, but helpful I would think
And, yes, I do agree that an external site, such as Waywardwaymarking is helpful, but not a substitute for what is on the official Groundspeak site, and that could certainly be improved. There are any number of experienced waymarkers that would be willing to contribute if asked.
Good thoughts. Thanks.
|
|
-
08-18-2009, 9:50 AM |
-
chapterhouseinc
-
-
-
Joined on 11-05-2006
-
SWEPMT
-
Posts 1,795
-
-
|
Re: Does Peer Review Really Work?
why not a minimum number in a set number of different categories (a la lucky 7 style).....
-A historic marker is a historic marker
-a fast food place is a fast food place
-most AGS points are NRHP points or NRHP contributing
-how many different statues are there
-benchmark
-ISS sighting
yes, there is a wide array of category posting requirements--some very lax, others very strict. requiring someone to participate in each of those will show the many facets of categories and how they operate......
|
|
-
08-19-2009, 5:38 AM |
-
dinoprophet
-
-
-
Joined on 07-07-2008
-
Orion (OR-ee-un), MI
-
Posts 302
-
-
|
Re: Does Peer Review Really Work?
chapterhouseinc:why not a minimum number in a set number of different categories (a la lucky 7 style).....
-A historic marker is a historic marker
-a fast food place is a fast food place
-most AGS points are NRHP points or NRHP contributing
-how many different statues are there
-benchmark
-ISS sighting
yes, there is a wide array of category posting requirements--some very lax, others very strict. requiring someone to participate in each of those will show the many facets of categories and how they operate......
I know those are just examples, but some of those are very limited in some areas. I'd feel obligated not to waymark any more historical markers in my area for fear of not leaving any for n00bs to fulfill the requirement. In geocaching, I'm opposed to a minimum participation requirement. I think it's more important here. Only a handful of people will ever see a bad cache; every waymarker is affected by a bad category. What about this: you must have X number of waymarks in a supercategory before you can create a subcategory there. Too arbitrary?
|
|
-
08-19-2009, 7:31 AM |
-
TheBeanTeam
-
-
-
Joined on 11-03-2006
-
Willamette Valley
-
Posts 1,748
-
-
|
Re: Does Peer Review Really Work?
I know these are just ideas but I don't prefer any participation requirements at all. I do prefer a forum requirement before a category is brought to peer review or a "workshopping" session that is a pre-review review.
|
|
-
08-19-2009, 11:49 AM |
-
chapterhouseinc
-
-
-
Joined on 11-05-2006
-
SWEPMT
-
Posts 1,795
-
-
|
Re: Does Peer Review Really Work?
yes, but there are new historic markers erected every year....and i dont think any state has maxed out yet....
|
|
-
08-20-2009, 6:37 PM |
-
the blue quasar
-
-

-
Joined on 11-04-2006
-
St. Catharines, ON
-
Posts 2,186
-
-
|
Re: Does Peer Review Really Work?
We are getting quite off topic. Sounds like a good time for someone to start a new thread about the recent ideas to possibly contribute articles or maybe even content for the weekly Groundspeak Newsletter.
Getting back on topic, I found myself thinking about this while driving around the other day....
How Peer Review Works:
Peer Review consists of a queue available to all Premium Members allowing them to review, vote, and comment on category hopefuls. Voting lasts for 3 days and is completely anonymous. In the case where a category is on the fence, Groundspeak reserves the right to make the final decision, but in most cases we will err on the side of approving the category. One of the things to keep in mind when voting on categories is that some people have a different idea of what is "wow!" If you think a number of people will enjoy the category, even if you won't personally, you should vote "yea" and perhaps give some recommendations for improvement.
There is a wording there that might actually apply. It says "a number of people will enjoy" which could be interpreted as "would a majority find it of value"? That is a two part thought... it could be asking us to vote as if we are speaking for what we believe most people would like.
Example: Most people would enjoy a category for "Take a Photo Locations" (like Kodak Photo Spots) and even if I didn't I would still vote for it.
Example: Very few people would enjoy a "Canon Dealerships" category, and even though I might since I work for Canon, most people have no use for such a category. In that case I should vote against it, even if I happen to like it.
BQ
|
|
-
08-21-2009, 6:08 PM |
-
0ccam
-
-

-
Joined on 11-09-2006
-
-
Posts 1,313
-
-
|
the blue quasar:Example: Very few people would enjoy a "Canon Dealerships" category, and even though I might since I work for Canon, most people have no use for such a category. In that case I should vote against it, even if I happen to like it.
BQ
Oh, I probably didn't tell you. You guys finally came out with a green-colored PowerShot, and I bought one for my wife for her birthday. Now I need you guys to come out with a PowerShot superzoom that does GPS and phototagging. Actually, having one that can take the range into account would be innovative. We now return you to your regularly scheduled Peer Review Thread.
|
|
Page 2 of 2 (46 items)
2
|
|