Born and reared in Pennsylvania, Ivory went West at the age of 21 to seek his fortune in the pine forests of Wisconsin. After five years in the lumber business he returned home.
In 1858, Ivory traveled to San Francisco via New York and Panama. After a short while he relocated to Contra Costa County, on a place known as the "Cook Ranch" in Green Valley.
Ivory conducted the fourth and last legal hanging in the county in the year 1874. Elizabeth Eichler, a "homicidal maniac", was intent on killing her husband, Valentine, and enlisted the assistance of hired hand Charles Martin to get the job done. To ensure his cooperation, Elizabeth convinced Martin that her husband was mistreating her and moreover, that she loved Martin and wished to go away with him.
After several attempts to poison and a plan to shoot Valentine failed, Martin grew disinterested in Elizabeth and her schemes. It was during a severe argument with her husband that Elizabeth finally took an axe to his head and embedded it in his skull. Per her instructions, Martin participated in an unsuccessful cover up story.
Before his execution, Charles Martin proclaimed on the scaffold, "Gentleman, I am here on this platform to die an innocent man. The woman deserves ten times as much to die." Elizabeth Eichler never stood trial. Two court-appointed physicians pronounced her insane and she was admitted to Stockton State Hospital where she died ten years later.
After his terms as Sheriff, Ivory was appointed by the Clay Street Bank of San Francisco as Superintendent of their 13,000-acre ranch known as the "Marsh Grant".