deleted deleted • over 3 years ago
Disappointed DevPost does not follow its own rules:
in your mail:
Submission Checklist: Do you have everything? -Amazon Alexa Skills Challenge: Aging & Engaging
They clearly said: "Reminder that any content over 3 minutes will not be viewed by judges"
however, the video of the "winning" skills: Brain Games, Memories, FingerString Appointments, have a duration of 4:42, 4:00, 3:06, minutes, that is, they should not even have been evaluated, much less winners . It is arbitrary the advantage with almost 2 minutes more than the rest. Completely disappointed in the lack of seriousness they put into their contests.
Comments are closed.
7 comments
Sumit Raj • over 3 years ago
They didn't even give the aws credits to the participants who at least submitted the project.
I couldn't complete my project due to lack of AWS credits which I kept asking them but didn't get it
David Brown • over 3 years ago
I usually wouldn't be one to complain in this regards, but I have to agree with you here Edgar. My original and more professionally put together entry video that I spent days on was flagged on the morning before the deadline as it didn't showcase enough of the actual use, but a video that clearly violates the entry requirements by a full 2 minutes wasn't, let alone ends up winning. I rushed together on the last day our final video and it still came in at 2 minutes 59 seconds...
It's also a little disheartening that some entries have even been allowed into the contest, let alone won a prize. Certain entries were originally entered into hackathons 2 years ago then just added a tenuous at best "addition" to their existing product, others are established businesses and had software in development for 3 years or more. I've been a part of hackathons for many years and allowing existing products quite frankly isnt in the spirit of what a hackathon should be about.
Overall this whole process has left a very sour taste for Devpost and Alexa Skills. The judging criteria if Im being brutally honest haven't been adhered to at all. "Including the uniqueness, creativity, and ingenuity of the Skill, originality," and "Does the skill have the potential to scale" are laughable at best. I don't mean to disparage other entries or come across as a sore loser but I, like many of the other entrants, will feel understandably disappointed without further clarification from the hackathon judges.
Pete Whelan • over 3 years ago
Hey Edgar and David,
I share your pain in losing and I actually edited my video because it was exactly 3 mins - so I made it under 2.59 just in case of disqualification! haha, that's funny now I look at it! But my take on it is this:
There is some ambiguity in the wording around the video. On the splash page, it says a video of "around 3 mins" while the full rules say under 3 mins. It could be argued that 4 mins are around three, it's not 15 mins after all etc... In addition, who's to say the judges simply did not watch any more than 3 mins of a video and then turn it off? While the rules also say that the judges don't have to actually use the skill but can just base decisions on the project gallery proposal, and there would therefore be a potential advantage to those who submit longer videos, I personally don't think this was the case. There were only 85 entries and heaps of judges. I personally saw lots of activity from 2 users in my stats during the judging period. I think the judges will have checked them all out - and no matter how good your presentation is, at the end of the day, it is about your skill, not the video. I think the takeaway in the future is to check for any ambiguity and ask for clarification in advance.
I agree that it's a little against the spirit that previously developed skills could be entered with a few tweaks, but again that's what the rules allowed. We all entered the competition knowing that. One-person developer entries like mine also have to compete against teams. I know I have to rely on coming up with a good idea, as I'm going to come up against some much more polished entries- but the old saying is you can't polish a turd! So it was somewhat pleasing to see that the grand prize was taken by a solo operator. All you need is a simple focused idea that is well on point and executed well!
The guys with the long video, team and upgraded skill have no doubt spent many man-hours and much money on their skill and it shows. It's clearly a good skill and I think it was judged on that rather than the video - it would be a shame to see it disqualified for that. So congratulations to them on being able to get back some of their development costs.
I think you guys both fielded strong entries and the next steps should be looking at how you can monetize the skill. I was particularly impressed when I saw Edgar's use of Sumerian - very creative and totally unique in the Alexa space - but I wondered how that worked in a voice-only environment. Perhaps you dropped a few marks there? If you ever want to collaborate on a Sumerian project let me know - I'd be keen, I've used Lex with it before too!
Cheers guys and take a moment to be proud of what you accomplished and learned in a short space of time!
David Brown • over 3 years ago
Definitely proud of what we've achieved in a matter of true weeks, late nights and weekend meetups, and hoping to go on from here Pete to do exactly that. That launch pad for us would have been a huge deal but alas, it wasn't meant to be this time! Maybe in a couple of years, huh? >o<
Again, not getting into the semantics around what constitutes as "around" but a +/- 10% would be what I would expect just like with speeding violations personally! Anything outside of this should have been disqualified.
"Oh I really want to finish this sentence and feature... 3:15 _should_ be ok, right...?" - Fine.
"Oh, but my skill is the best and super professional! I'm going to use another 75% over my time allocation to really show off all it can do!!" - Not fine. At all.
As I said, I got an email about my original "super shiny" video on the final day, then recorded a new entire 45 minute full walkthrough of my app, then managed to condense down to 2:59:99 (which ended up uploaded to 3:01 anyway, lol!) on the nose, all in the last day and I did this as a side project. Given this is clearly some of the entries full time efforts, I'd expect the same level of courtesy from all entrants
In any case, a huge congratulations to all that entered and some really positive use cases for Alexa Skill for an age group that really does deserve some love and attention given the state of the planet. Hopefully we can personally learn from this experience and make a better and more caring app with or without the support from the Skills team, and I do wish all developers the very best for whatever direction their apps take.
Janet Fang Manager • over 3 years ago
Hey everyone,
Thank you for your patience as we work through this. We have communicated your concerns with the Amazon Alexa team and also discussed within our team at Devpost. We understand your disappointment on the video length, it was not our intention to mislead anyone in the one-week-out reminder. The official rules and the overview tab however, do state that the video portion of the submission “should be less than 3 minutes” or “about 3 minutes”. Demos are important but not the only materials judged.
We supported 2 rounds of judging and the judges really took their time deliberating every part of the submissions. We sincerely value all of the hard work that was put into each project and thank you all for being part of the community. We truly wish everyone could win every hackathon.
David Brown • over 3 years ago
Sorry Janet, I'm going to be quite blunt here, but all I’m reading is a whitewash after the horse has bolted. I'm not some petulant child that needs everyone taking part to win some kind of medal. Winning and losing is a part of life, learning to handle defeat in an honourable way and growing from it is an important lesson to learn, but that's only if there is a level playing field to begin with for all participants.
The deck, in this instance, feels like it was stacked from the start. There is nothing more evident than the fact that the majority of the 9 winning entries have been working on what they ultimately submitted for their Skill for several years at this point.
I appreciate this wasn’t against the rules, but you’ve specifically addressed the ambiguity over the meaning of the word "should" but failed to address the "must" of if the Entrant’s Project existed prior to the Hackathon Submission Period, it must have been significantly updated after the start of the Hackathon Submission Period. A bit of due diligence on the aforementioned winners would have disqualified at least one and likely rose a number of interesting questions about a few others.
Personally I do think submitting an almost 5 minute video when it clearly states it must be under three is an insult to the other entrants and shows a complete lack of respect to the judges, but then again I'm also confused as to how increasing the length of the internal game timers is a "significant upgrade" and not an insult to the elderly.
I’ll keep plodding on with our Skill regardless of this so I appreciate the opportunity to have our ideas reviewed at this stage. Hopefully another competition will come up in a few years so I can rebrand it to fit the brief, then submit a 12 minute music video when it clearly asks for a 5 minute elevator pitch...
Joel Ramey • over 3 years ago
Just popped in to see if anything ever came of this, but I'm not seeing an update. Was the matter ever resolved, or is it just staying as is?