From the user testing, I realized that the directions were still unclear for my purpose. To change this, I played around with iconography and incorporated Google map directions in text form. The user can also go back to view the map on the upper right corner if desired.
The new map will display the directions one by one, and as the user reaches the goal, it will turn a different color. The directions will keep scrolling up until the user has reached the destination. The microphone icon I had before seem to confuse people (they thought its function was to speak into the phone), so I changed it to a Speaker.
Digital Prototype: