To be fair, Mr Pistone, a private in George Patton's army, never thought of his act as theft. He just needed proof he was there.
"I thought who the hell's going to believe I was in Berchtesgaden?" he said. "I'm going to need some proof."
As the Americans raced across southern Germany in the spring of 1945, Mr Pistone recalled, the soldiers were intent on capturing as many German soldiers as possible. And finding Hitler.
"They were giving up like mad by that time, but we were looking for Hitler. Because they said he was still alive," he said.
And so the young John Pistone found himself walking through the gates of the Berghof, Hitler's mountain retreat near Berchtesgaden, in the Bavarian Alps.
"We had a feeling like we just missed Hitler," Mr Pistone remembered. "It seemed like... someone had just left in a hurry."
The place had already been stripped bare by other American souvenir hunters, but in a cabinet Mr Pistone found a large photo album, full of immaculate black and white reproductions of paintings.
The 'Hitler Book'
He had no idea what it was, but he thought it looked interesting and would do nicely as a memento. For the next few months, until he made it back home to Ohio, he lugged the volume around.
"That damn thing was heavy! But I was determined to get it home," he remembered.
For decades, the album sat on John Pistone's shelf, brought out only to show family and friends. His children called it "the Hitler book".
But it was not until Mr Pistone decided to install a washer and drier in an upstairs bedroom that the book came to the attention of a local history buff who, in turn, contacted Robert Edsel, author and president of the Monuments Men Foundation.
The Monuments Men were a group of some 345 men and women from 13 countries who scoured Europe during and after WWII, looking for artistic and cultural items stolen by the Nazis.
When he heard about the album, Mr Edsel figured he knew what it was, but he flew from Texas to Cleveland to make sure.
"When I first saw it, there was little doubt in my mind about it being authentic," Mr Edsel said. "But the question was, as always, where did it come from?"
Examination confirmed Mr Edsel's initial hunch that the book was one of 31 albums that formed a catalogue featuring art selected by Hitler for inclusion in a huge National Socialist museum of art, planned for the Austrian city of Linz.
The museum, had it ever been built, would have included looted masterworks from across the continent, but Mr Pistone's album, Number 13, mostly consists of reproductions of little-known German and Austrian 19th Century paintings.
Decision time
A triptych by Hans Makart, The Plague in Florence, was a gift from Hitler's Italian ally, Benito Mussolini.
When Hitler's initial efforts to acquire the work from an Italian banking family in Florence were rebuffed, Mussolini confiscated their entire villa and gave the paintings to Hitler, who apparently reciprocated by sending the Duce a bust of himself.
Another work - Frederick the Great Travelling, by Adolf Menzel - was one of Hitler's favourites and used to hang above his desk.
John Pistone was reluctant to part with the "Hitler Book", having held on to it for so long, even though Makart and Menzel meant nothing to him.
But when he read Mr Edsel's book about the Monuments Men, he decided that the time had finally come to part with the souvenir from Germany.
"I talked to my wife and I said 'I think we're going to put the book in his hands,'" Mr Pistone said.
"I feel very, very good about it, now that I know that it's important to people other than myself and my family."
At a ceremony at the State Department in Washington last week, Mr Pistone handed the album over to the German ambassador.
After a brief appearance at the National World War Museum in New Orleans, the album will be sent to Berlin, to join 19 of the original 31 volumes. The hunt goes on for the remaining 11.
And for John Pistone, there's a feeling that he has done the right thing.
"Life has been so good to me," he said. "I've been married 60 years. Five children. Ten grandchildren... And when you leave this world, it's not how much money you leave. It's setting an example. I hope this sets an example. For my children."
