AI Education Matters: EAAI mentored undergraduate research challenges past, present, and future

by Todd W. Neller and AI Matters

04 December 2020

2014/2015: Parameterized Poker Squares

In 2014/2015, the first mentored undergraduate challenge was called The EAAI NSG Challenge: Parameterized Poker Squares. DARPA had a history of Grand Challenges, so our acronym “NSG” was a tongue-in-cheek modest nod to DARPA’s Grand Challenges, noting that undergraduate research challenges, while non-trivial, would be Not-So-Grand.

Poker Squares is a folk sequential placement optimization game. Using a shuffled 52-card French deck, the rules of (Morehead & Mott-Smith, 1949, p. 106) read as follows:

Turn up twenty-five cards from the stock, one by one, and place each to best advantage in a tableau of five rows of five cards each. The object is to make as high a total score as possible, in the ten Poker hands formed by the five rows and five columns.

Two predominant scoring systems, American and British, assign point values to achieving row/column Poker hands. Parameterized Poker Squares was a generalization where the scoring system was supplied to players at the beginning of a game and players were challenged to form play strategy under time constraints.

In our contest, AI players were given 5 minutes to learn play strategy from a given scoring system, some fixed and some generated at random within different design constraints. Players then were required to play such strategies for 100 unknown shuffled decks with a time limit of 30 seconds for reasoning during the play of each deck.

The contest drew participation of seven teams with mixed results and four papers accepted for publication through peer review.

2016/2017: Widely-Accessible AI Robotics Tasks

Zachary Dodds organized the second mentored undergraduate research challenge, Widely-Accessible AI Robotics Tasks. Rather than create a competition, each design challenge entry was awarded an accolade for a unique virtue of its contribution. Contest details focus on three design trade-offs:

AI sophistication: more is better
Wide adaptability: easier is better
Compelling contexts and toolsets: the more creative and more authentic, the better

The challenge attracted six entries with corresponding abstracts/papers and presentations at EAAI-17.

2018/2019: Birds of a Feather

Birds of a Feather was offered as a research challenge in this AI Education Matters column in the summer of 2016.

The 2018/2019 mentored undergraduate research challenge was for students to research some aspect of this newly-designed solitaire card game and publish their research. Most students focused on search solution efficiency and state unsolvability detection/prediction. This resulted in seven accepted peer-reviewed papers and one poster.

Current 2020/2021 Challenge: Gin Rummy

Gin Rummy is one of the most popular 2- player card games played with a standard 52- card deck. Ranks run from aces low to kings high. The object of the game is to be the first player to score 100 or more points accumulated through the scoring of individual hands. In this Gin Rummy mentored undergraduate research challenge, students develop AI players to play games within a time constraint of 30 seconds per player per game (consisting of multiple hands) using North American gin and undercut bonuses of 25 points each.

At time of writing, 14 teams of 50 total mentors and undergraduate students have submitted AI players for competition evaluation. Corresponding papers are already trickling in more than a week in advance of the submission deadline. This is our highest participation rate to date and bodes well for the future of these endeavors.

Looking Forward

It is clear that there is a consistent recognition of the value of mentored undergraduate research opportunities in our AI Education community. Perhaps the next best step is to establish a committee and workflow for consistent offering of diverse challenges each year with distributed and rotating responsibility. At time of writing, we are seeking volunteers to serve in organization, promotion, mentoring, and peer-review in this cyclic process.

Because the challenges have largely been organized by one primarily interested in Game AI, challenge problems have not tapped many of the diverse subfields of AI research. To engage a broader cross-section of faculty mentors and undergraduate students, it is necessary to bring in additional faculty expertise and passion for designing mentored undergraduate research challenges.

One possible improvement would be to design and announce the next challenge well before the beginning of the academic year. This would allow students in both academic year courses (e.g. capstone research) and summer research programs to participate before the following fall deadline. While it is too late for this academic year, we envision a future organizational structure that would establish a regular calendar workflow to distribute the considerable work necessary to create and support these endeavors annually going forward.

Conclusion

Looking at the past and present, there is a consistent level of energy and engagement around the worthy task of mentoring undergraduate research skill. Our greatest challenge looking forward is to establish an organizational structure that features diverse research topic areas from year to year and will operate with an annual rhythm that allows for both academic year and summer participation. If you are interested, please email the author of this column with your CV, website, and a brief description of your interests in supporting our endeavors.