How Is AR Development Accomplished In Unity?

I’ve recently decided to develop an iOS application to learn more about AR development in Unity. This blog will be used to take notes and document design decisions I must make. Let’s get started.

I began my journey on Ray Wenderlich’s website. This Ken Lee AR Foundation tutorial on the site was a great way to find necessary resources and to understand more about Unity’s unique approach to merging all AR SDKs into one with AR Foundation.

After completing the tutorial, I have the following AR features running on my iPhone: AR Plane Manager that detects surfaces through the device’s camera, AR raycast logic that detects the player’s touch input, and simple “Doodle” logic so the player can draw lines of different colors and thicknesses on the detected AR surfaces.

The app’s Doodle logic will not start until the camera detects an AR surface.

The app’s Doodle logic will not start until the camera detects an AR surface.

The Doodle logic allows the player to choose the color and thickness of the line they can draw with their finger. It also clears all lines with the “Clear” button on the right.

The Doodle logic allows the player to choose the color and thickness of the line they can draw with their finger. It also clears all lines with the “Clear” button on the right.

The tutorial finishes up with some post-processing effects like the vignette effect visible in this screenshot.

The tutorial finishes up with some post-processing effects like the vignette effect visible in this screenshot.

While testing the app’s builds, I ran into my first problem. The UI at the bottom of the screen features a slider for the player to choose the thickness of the line they draw. Using this slider would often trigger my phone’s app-switching feature, so I decided to move the UI to the top of the screen.

The slider often switches the AR app to the previously used app.

The slider often switches the AR app to the previously used app.

So the UI is moved to the top of the screen for now.

So the UI is moved to the top of the screen for now.

Next, I would like to be able to draw on my face, so I want to implement a “Camera Flip” feature and face tracking features.

Unity’s AR features are quite buggy for me in 2020.2.2, so I have to keep uninstalling and then reinstalling some XR packages just to get basic XR features to appear. This is troubling when I try to build and magically the XR features are missing. The built app launches, but does not automatically fire up the rear-facing camera as it used to.

On the subject of cameras, Unity’s AR Foundation uses an enumeration (CameraFacingDirection) to choose between no camera, world camera, and user camera. World camera is the rear-facing camera, and user camera is the front-facing camera. I added an AR Camera Manager and a simple script to try to control the device’s camera, but to no avail. More research needed to debug this feature a bit more.

Screen Shot 2021-09-27 at 12.20.43 PM.png
Screen Shot 2021-09-27 at 12.25.23 PM.png

Hopscotch Hamlet

The first assignment in Jesse Schell’s Game Design course allowed me to understand my design process better and learn clear, concise ways to format my design documentation. I am sharing this on my blog to remember one of my favorite course’s best (and first) assignments. Warning: Some minor profanities and legal-aged topics ahead…

Assigned: Tues. Jan. 15, 2019

Due: Tues. Jan. 22, 2019

PART I:

Jan. 16, 2019, 5:56 PM; Jan. 19, 2019, 2:11 AM; Jan. 21, 2019, 6:31 PM 

-Hopscotch requires skill, endurance, and precision

-Hopscotch is great for those with shorter legs where jumping longer distances is a challenge.

-Hopscotch games tend to be pretty quick

-Hopscotch is simple to play

-Hopscotch has the thing you’ll be doing most in its name (scotching)

-Hopscotch has a little bit of chance and a little bit of luck depending on your thrower

-Hopscotch feels dated

-Hopscotch is seen as a “child’s game” by most

-No real sense of loss (apart from schoolyard shame)

-Hopscotch is really repetitive, how does it end?

-What do the other players do while the one person is playing? Just watch?

-Drawn square “board” is quite basic

-Maybe a mechanic where the standby people must do something to keep entertained? Or to get their turn back?

TLDR;

How can a broader age group enjoy and even reconsider the term “Hopscotch”?

How can Hopscotch become more competitive, with a stronger sense of urgency?

Can Hopscotch include everyone in the room rather than one player at a time?

Generate 50 one-phrase ideas (but I will shoot for 100!).

Jan. 16, 2019, 8:14 PM

  1. HopSCOTCH drinking game

  2. Hopscotch but you must yell out an embarrassing truth about yourself

  3. CrabScotch walk backwards on hands and feet like a crab

  4. Hopscotch but it’s a massive Christmas chocolate calendar

  5. Hopscotch but the floor is LITERALLY lava

  6. Hopscotch but the ground is covered in larvae

  7. Hopscotch but instead of a rock or whatever the fuck you throw on the ground, you must eat a Fear Factor (rip) worthy gross bug or animal testicle or something lol

  8. Hopscotch but there’s LEGOs on the ground

  9. Hopscotch but there’s old gross gum on the ground and it’s really gross because your shoe gets caught on it every time you jump

  10. 1Hopscotch but an XR rhythm game

  11. Hopscotch but the double pieces slide like those slide-puzzles and it’s fun because DANGER

    12.Hopscotch but it’s in the year 3000 and it’s gone from children’s game to shipping dock worker game and it’s played on the side of shipping containers with magnetic boots (year 3000 remember?)

    13.Hopscotch but as Frogger and you must also avoid cars (I just got scared by a car because I was looking down at my phone)

    14.Hopscotch but Russian roulette style and one block per round is a vortex to another dimension (bye) and it would be process of elimination I guess?

    15.Hopscotch Tetris? How you ask? Well, everyone has a GameBoy color while playing Tetris and just throws their GameBoys instead of rocks and whoever’s GameBoy lasts the longest wins

    16.Hopscotch on ice?

    17.Hopscotch but it’s a life sized Life board and you just play the game if life on it

    18.Hopscotch but while dressed in clothes two sizes too small for you (ouch)

    19.Hopscotch but as a giant frisbee golf course

    20.Hopscotch but as a giant ski/snowboard mountain

    21.Hopscotch but played off of a diving board into a pool (running start NOT recommended)

    22.Hopscotch but played at JumpStreet off a trampoline into a foam pit (running start required)

    23.Hopscotch where you play with shadows on a wall with your hand

    24.Hopscotch but desk sized and played with fingers (so cute)

    25.Hopscotch for smokers circle where you flick your cigarette butt into the blocks and hope for the best while lighting the next one up brooooooo

    26.Hopscotch that teaches French recipes

    27.Hopscotch scratch-n-sniff that makes up funny Mexican food combos (Taco Bell does it with like 4 ingredients sooooo)

    28.Hopscotch Go: played on a global scale (Pokémon go) and the first winner gets a huge prize (Curiosity iPhone game)

    29.Hopscotch but every square is an M. Night Shyamalan movie that you HAVE to watch when you land on it. Sometimes you’ll get The Sixth Sense, sometimes you’ll get Avatar.

    30.Hopscotch but it’s a giant mad libs board and every square you land on is a secret

    31.Hopscotch but your “shooter” is a football

    32.Hopscotch but it’s made of scrambled LEGO floor

    33.Hopscotch but you just play to the beat of Keep on Lying by Tame Impala

    34.Train track hopscotch

    35.Hopscotch but as a flick-ball desktop game

    36.Musical chairs hopscotch

    37.Beer pong hopscotch

    38.Butterscotch-lined HopButterscotch

    39.Play testing for toddlers

    40.Agescotch every cube has an age you must become (wizards play this) and transform into for a week and who ever is most successful after the week wins (success is voted on)

    41.Improv scotch


Jan. 17 2019, 12:03 AM

42.Hopscotch but you must play in a swimming pool

43.Hopscotch but you must play with pool floaties on. No pool? Sorry.

44.Hopscotch, on horses.

45.Hopscotch played with tazers...?

46.Hopscotch but cliff diving form? Super dangerous...

47.Hopscotch but you must build the Hopscotch board as you go using tape.

48.Hopscotch but you must rip off the board as you go.

49.Hopscotch CAPTURE THE FLAG?

50.Hopscotch tug-o-war

Jan. 19, 2019, 5:45 PM
51.Hopscotch slay the spire (digital version)

Brainstormed with Alison via FaceTime on Jan. 19, 2019, 11:13:

52.Truth or Dare Hopscotch (Newlyweds game?)

53.Spin the bottle hopscotch

54.Mini-trampoline hopscotch

55.Texture changing hopscotch

56.Multitasking hopscotch

57.Shoot something at the same time

58.Slingshot hopscotch

59.BB gun hopscotch

60.Paintball hopscotch

61.Try not to get hit by paintball hopscotch

62.RTS Hopscotch

63.Hopscotch blindfolded

64.Hopscotch but reversed and upside down (glasses on Grand Illusions)

65.Hopscotch on rollerblades

66.Hopscotch roller derby

67.Ice skating hopscotch

68.Hot potato hopscotch

69.Baked potato hopscotch where you choose your own potato ingredients

70.It’s how you order food at a restaurant Hopscotch (like chipotle but for people who can’t make up their minds).

71.Random food generator.

72.Hopscotch to open a secret entrance.

73.It’s an AR Oculus AR kit where you choose your movement with a wrist-attached bean bag essentially. Bungee corded design that simulates actual throw’s angle. Return back to sender and reset.

74.Hopscotch while zombies chase you. Survival mini game?

75.Dancing Zombie mini game

76.Nail painting hopscotch to help you choose nail paint color.

77.Drinking water while hopping hopscotch

78.Balancing egg on a spoon hopscotch

79.Juggling hopscotch

80.While throwing axes

81.Hopscotch underwater

82.Lake hopscotch where you have to jump into it with a rope.

83.Lake hopscotch where it’s a project semester trip and you use it in Jesse’s Game Design as an adventure module type trip!

84.Glow in the dark hopscotch

85.Galaxy hopscotch in VR (Roll a planet instead of a rock, or a meteor)

86.Hopscotch in hell

87.Hopscotch videogame trivia (Must answer trivia question to move on?)

88.Hopscotch Jeopardy version?

89.Hopscotch wheel of fortune

90.Hopscotch price is right

91.Hopscotch family feud

92.Hopscotch during an earth quake

93.Hopscotch as an 85+ year old person?

94.Hopscotch Jackass

95.Hopscotch Bandersnatch episode choice

96.Hopscotch while doing your makeup

97.Melted chocolate is coming at you from all directions hopscotch

98.Hopscotch while lucid dreaming

99.Hopscotch AI

Jan. 19, 2019, 11:22 PM

100.Gravy hopscotch

101.Foot squeezed wine hopscotch

102.Hopscotch bowling

Jan. 21, 2019, 6:07 PM

103.Hopscotch with bouncy ball passing

104.Hopscotch with a whiteboard involved somehow

105.Hopscotch slip-n-slide


Part II:


Jan. 21, 2019


Digital:

10. Hopscotch but an XR rhythm game

    1. Each hopscotch square is either a horn, a drum type, or a sound effect

    2. Beats could either be schoolyard friendly or just some sick-nasty beats worthy of DJ Hero (RIP)

    3. Wires would be a problem, so google glass type futuristic AR headset necessary…

28. Hopscotch Go: played on a global scale (Pokémon go) and the first winner gets a huge prize (Curiosity iPhone game)

    1. Square shaped global landmarks and states (like Colorado or Central Park, but with further research to make it as “fair” a globe-trotting game as possible)

    2. To play, one must hopscotch once per landmark across the globe (10 total landmarks)

    3. The winner gets to hopscotch against the Harlem Globe Trotters (get it?) and a full reimbursement of travel. Sorry losers.

86. Hopscotch VR played on a universal scale.

    1. Surrounded by stars and empty Space

    2. Squares are galaxies

    3. Your thrower is a planet, and every galaxy can change color and variation changes based on your step


Aforementioned problems with Hopscotch (I added a few while working on Part II):

-Hopscotch feels dated

-Hopscotch is seen as a “child’s game” by most

-No real sense of loss (apart from schoolyard shame)

-Hopscotch is really repetitive, and how does it end?

-What do the other players do while the one person is playing? Just watch?

-Drawn square “board” is quite basic

-Maybe a mechanic where the standby people must do something to keep entertained? Or to get their turn back?

-How can I create urgency in Hopscotch?

-How can I add spontaneity into Hopscotch?


Physical:

30. Hopscotch but it’s a giant mad libs board and every square you land on is a secret (mixed with improvisational acting)

    1. Start with 3 strips of paper per player.

    2. On the first strip, write down a famous person that every player would know (this is called an “A” Block). On the second, write down a strange personality quirk (only speaks in high-pitched voice, or must maintain eye contact with player X, for example). On the third, write down an embarrassing thing someone would say (“I can’t read,” or, “My left hand is smaller than my right.”).

    3. Using tape, attach each strip to a Jenga block or domino piece

    4. Players randomly throw their blocks onto the board, and as they play they acquire a persona that they must imitate while hopping or not, a quirky personality trait, and a reason for their actions.

79. Balancing egg on a spoon hopscotch (or a ball)

    1. Hopper is balancing an egg/ball while hopping

    2. Other players are trying to knock the ball away with playing cards or other SOFT object that wouldn’t hurt too much if it hit the hopper

    3. Process of elimination until someone makes it across

    4. I’d like to stress how tempting it may be to hit someone with a Jenga block so both the honor system and underhand throwing are implied

102. Hopscotch Bowling

    1. Hopper bowls small bouncy ball towards 6 diamond shaped setup of Jenga blocks

    2. While hopping through blocks, other players must race to reorganize 6 of these blocks before hopper returns.

    3. Whenever others finish reorganizing, they yell, “STOP”. Hopper MUST stop in tracks and jump back to start point from where they are.

      1. If they cannot make it, they are eliminated from the game, but can still compete in Jenga reorganization.

      2. ii.If they CAN make it, reshuffled into line and others try again to eliminate

Asking players to be on their celebrity impression A-game was going to be a lot to ask of tired ETCers, so I decided to run with both Egg-Balancing (now repurposed to a ball for clean-up sake) and Hopscotch Bowling.


Egg-Balancing Hopscotch was an instant “shoot your babies” moment. After just the first pass I got the sense that my players were not excited, and I failed to quickly invigorate it in any way, so I moved on with Hopscotch Bowling.


Part III:

Hopscotch Bowling

Players: 4-6

Time: 5-15 minutes

-The person who ate most recently goes first as the Hopper. |

-All other players take 6 Jenga blocks and place them in the shape     | |

of bowling pins (see diagram to the right) in the final space of  

a Hopscotch board.

-Make sure all blocks are standing upright but won’t fall over too easily!

-When all other players have their Jenga “pins” standing, they kneel behind their respective formation of blocks, becoming Kneelers.

-When all players are ready, the Hopper bowls a bouncy ball towards the pins, in hopes of knocking as many as possible over.

-If the Hopper misses every single pin, they get a strike.

-Three strikes, you’re out, and will never be a Hopper again.

-If you knock over some pins, you must Hopscotch with only your dominant foot on single blocks, both feet on double blocks, then back to only your dominant foot on single blocks. 

-While the Hopper hops, the Kneelers must help each other out to rebuild the broken formations back into the aforementioned diagram. 

-When all formations are replaced on the board, the Kneelers yell, “STOP!”

-The Hopper must stop dead in their tracks and can now use both feet to jump the remaining distance to the starting bowl point.

-If the Hopper makes it to the beginning with both feet firmly planted, they become a Kneeler and switch out in hopes of knocking out the next Hopper.

-Repeat until all but one Hopper has been eliminated.

Time For Bed Dev. Log (Version 4)

Time For Bed was made in 48 hours by 6 people during Global Game Jam 2019 - Pittsburgh. There, it took home the award for Jammer’s Choice. a few months later, Swapnil Mengade, lead programmer of the project, submitted the game to the TOKYO SANDBOX game show, where it got selected for showing. COVID postponed the show, but we have since decided to port and optimize the 2-day game jam game into a more full-fledged release for both PC and mobile devices, prioritizing mobile devices. The game’s systems all work as a playable prototype, but we plan to build more levels and game types into the final release, so a fresh project was the logical next step. We chose Unity 2020 and a target audience: Mobile.

Player Controllers

Understandably, the game jam version of Time For Bed was made very quickly with little time for discussion of future work on the project. As a result, this version of the game uses a monolithic Player Controller script. To the left, you can see a small snippet of the original PlayerController script.

Immediately, I realized that this PlayerController script actually controlled two types of players with a simple public bool check: isParent (seen here on line 21 and checked for on lines 52 and 56). If checked true in the Unity editor, the player GameObject is a Parent, and if unchecked, the player is a Kid.

Instead of one monolithic script, I decided to use this important bool check to separate this PlayerController into two separate ones: a ParentController script and a KidController script (see below).

In creating the two new Controllers, I also wanted to remove the input checks from Update that you can see on lines 50 and 51. Using Unity 2020’s new Input System, the ParentController and KidController will wait for the player to use a stick or push a button before pinging the rest of the script, rather than checking every frame for inputs from the player.

Player statistics like movement speed, parent’s grab range, kid’s jump velocity, and parent’s throw velocity (lines 16, 24, 31, and 39 respectively) were all great details we were able to implement as soon as we did during the game jam. Without the amount of testing we were able to get done for the game, these mechanics would not have felt as great as they did at the end of the game jam. In early discussions for the mobile port, however, Swapnil and I knew we wanted to have different types of Parents and Kids with unique move speeds, jump velocities, or grabbing/freeing ranges. For this challenge, I decided to use Scriptable Objects for all of the player statistics.

Parent can take in scriptable object variables, walk, climb stairs, detect when kid is nearby, grab kid, put down kid in bedroom.

For Kids, these stats are things like jump velocity and move speed. Kid can take in scriptable object variables, walk, climb stairs, jump, and wake up other kids.





Input System:
Unity’s new Input System is great and allows way more flexibility through Actions rather than waiting for a KeyCode press during every frame in Update. Instead, the new Input System allows you to dedicate all kinds of inputs from different devices like mouse/keyboard, controllers, and mobile devices into one single action you check for in your scripts. I’ve come to the realization that it allows for much faster growth of a project in case buttons need to be moved around, but the action will still stay the same in your code. I also decided to prototype a select screen because I couldn’t playtest both a parent and kid. I used a PS4 controller and mouse and keyboard for players one and two, respectively.

Main Menu

Note+Aug+27%2C+2020+9_52_33+PM.jpg

Cinemachine follow cams.

up/down functionality

This script takes one UI Canvas array and one Cinemachine Virtual Camera array. On player input, it cycles up or down through the arrays. Since they must always cycle together, a single integer (currentCanvasAndVCam) is used for both arrays. EDIT: After learning a bit more about the UI functions in the Unity editor alone, I decided to incorporate more editor functions to enable and disable different game objects within the editor. I use these in conjunction with the private int in the StartMenu script (now called currentMenuAndVCam)

Blend between cameras on player input (move action and back to the bottom floor when going from settings to play). Ask Swap Why I get errors and how to cycle between a list. Simple. Check for it.

For simplicity’s sake, Swapnil also recommended I convert both currentVCam and currentCanvas into one single integer. This new one is called _currentCanvasAndVCam.

For left and right functionality, I used Unity’s Navigation UI feature, explicitly telling each button where to go.

The Local/Online button toggles the Play button’s features. If Local is set, then the play button will take the player to the local multiplayer screen (Name, Host game, Available games nearby, or back to main menu). If online is set, then the play button will take the player to the online multiplayer screen (Name, Host [Create game], Public [find game], private game [enter code], or back to main menu).

Simple AI with state pattern (this pattern is used in computer programming to encapsulate varying behavior for the same object based on its internal state) (a little too advanced for me… Instead, I just used a simple animation of them walking back and forth until Swap gets AI up and running)

For the Settings menu, I wanted an audio mixer to control the volume settings. But it wasn’t actually working so I found this YouTube video which did the trick: https://www.youtube.com/watch?v=xNHSGMKtlv4

Set minimum value of sliders from 0 to 0.001. Added Mathf.Log10 because mixers are set to decibels.

Mobile Devices

Game Controller

The simplicity of the game made porting this one pretty simple. The only trouble I had was in Swapnil’s original implementation of AllCHildren Asleep. His read as backwards to me, but after asking him why he implemented that way, here is his explanation:

Loading Screen

Created an enum script for more user-friendly build index management.