They will now take on Korea in the bronze medal playoff on Saturday night, a team they lost to by a point during the group stage.
As was the case in that meeting, Japan's three-point shooting proved too much for the Gems as they hit 12 three-pointers, out-scoring Australia by 12 from beyond the arc.
Amazingly, Japan did not even attempt a free-throw for the game but their jump shots proved very effective while seven more offensive rebounds also helped them put up 17 more shots for the game.
On fire from the opening tip, Japan moved in front by 15 points in the second quarter after leading by nine at quarter-time.
The Gems responded by increasing the tempo and Agnes Emma-Nnopu (eight points, seven rebounds, five assists, two steals, two blocks) led the way with her trademark hustle and defensive pressure seemingly lighting a spark for the Aussies.
With Miela Goodchild (26 points, seven steals) and Ula Motuga (16 points off the bench) starting to fire offensively, the Gems began to reel in the deficit and Emma-Nnopu and Emma Clarke (eight points, seven rebounds) contributed as well as Australia put together a 21-2 run.
Incredibly, that gave the Gems the lead at half-time and with their forwards finding their range alongside Goodchild, Australia were matching Japan's three-point capabilities.
Unable to penetrate during that run, Japan quickly corrected in the third quarter though and they restored their healthy lead from earlier in the game as they went on a 20-4 run themselves.
The Gems dug deep and scored the first nine points of the final term to briefly bring the deficit back to single-digits but they could not put together one final big run as Japan handed Australia their second defeat of the tournament.
Find the box score here and watch the replay below.
Australia now face a re-match against Korea who got the better of them, 63-62, earlier in the week when they met in the group stage. Stay tuned to Basketball Australia's social media channels for tip-off time.
All games for the tournament will be live-streamed on the Basketball Australia Facebook page.
The Australian Gems Schedule for the 2018 FIBA Under-18 Asian Championships (all times AEDT):
October 28: Australia def. Indonesia (96-28)
October 29: Australia def. Chinese Taipei (106-66)
October 30: Australia def. by Korea (62-63)
November 1: Semi-Final Qualifier: Australia def. New Zealand (82-66)
November 2: Semi-Final: Australia def. by Japan (77-90)
November 3: Bronze Medal Playoff vs. Korea (Time TBC)