There never was a Grand Master Luke of the New Jedi Order. He isn't canon and thus never existed.
Misconception you and everybody else has: this scene
isn't what actually happened, it's the lie Kylo keeps telling himself. Luke thought he could kill his own nephew and prevent countless deaths, and instead found he was too good. He faltered. Which is
B. totally consistent with his past actions and Skywalker lineage.
You say that he was selfish and cowardly, but what Luke did goes beyond what Yoda and Obi-Wan did. The latters went into exile to prepare their comeback and renew the Light vs Dark fight once again; though they realized that most conflicts that the galalxy suffered from (eg. Clone Wars) came from Jedi vs Sith skirmishes who both ignored the civilian casualties, they still decided to mold Luke to be the next pawn in a mystical intergalactic chess game that spanned millennia.
Luke realized that by attempting to train Kylo Ren and other Jedi, by refusing to be The Last Jedi and thus respect the outcome of the Prophecy of the Chosen One, he had invited conflict, his equal in the dark side, in the form of Snoke, and the Knights of Ren; their rise to power is The Force attempting to Balance things out, and punish Luke for attempting to unbalance the galaxy. So he decided to retreat, force himself to stay out of the conflict, stop the cycle. He won't play the Force's twisted game anymore, for the sake of the Galaxy. When you see him in EP.7, you see the conflict in his face. He wants to grab his old lightsaber, he wants to ignite it. It would be so easy for him to jump back into the fray. But if he does it, then Snoke will eventually become more powerful as well, find an amulet or something, because the Force will have willed it so, etc. So he stays back. If he dies, the cycle ends; no Jedi, no Sith to derive from them.
Of course he was nothing like the original character. The original character is nothing but a boy, swinging a lightsaber playing Jedi. This Luke is more of a Jedi than the original will ever be, because "wars not make one great", and Old Luke understands that completely, as seen from his actions in the film, even from the way he dies. The Jedi way hinges on self-sacrifice for the greater good, which is exactly what Luke does.
What promise to Yoda? Pass on what you have learned? As previously explained, Yoda and Palpatine are two sides of the same coin. Of course they want the Jedi/Sith to live on, because neither of them give a shit about the little guy, not really; they just want the other gone from the face of the galaxy. Which is exactly the point Luke is trying to drive across to Rey: though the Jedi are glorified, they were dicks, just as much as the Sith, only on the other side of the spectrum. Rey doesn't need to be a Jedi to be a hero. She only needs a teacher who can show her how to hone her skills so she doesn't kill everyone around her, and from that perspective, the Sacred Jedi Texts are more than enough.
Finally, the general understanding is that Luke has spent 30 years in exile. Wrong. He spent 6, at most. That's less than Ben Kenobi did. Kirik Infil'a took a decade-long Barash vow, thus hiding during Order 66, and nobody says shit about him. Luke goes into hiding for 6 years and everybody bitches about it. If the galaxy went to shit after only 6 years, it means it was already well on its way whether or not he left.
My only regret is that we didn't get to see Luke's lightsaber skills more... that fight with Rey should've ended with him as the victor, not her standing proudly before him.