Before the automobile was invented, police officers patrolled on horses. The officer and horse were a team. The Mounted Unit plays an integral part of the San Jose Police Department with an emphasis on community policing.

The San Jose Mounted Unit began as a volunteer unit in 1970. Police Reserves, donating their time to the City and utilizing their own personal horses, patrolled the parks during summer weekends.

In 1982, several regular members of the Police Department joined the volunteer unit, and an agreement was reached with the City, whereby each member could board their personal horse at the old Tully Road Stables, and the city would provide for the feed and the shoeing of the animals.

By 1984, the volunteer unit had grown to ten regular officers and one Reserve officer, and was still a volunteer assignment, worked on a collateral basis in addition to each officer's regular assignment.

In 1985, three factors gave support to the formation of a full-time Mounted Unit: the renovation of the downtown area, the expansion of the City parks system, and the decision by the Council that Park Rangers would not be sworn personnel. In late 1985, an agreement was reached between the Police and the Parks and Recreation Department, whereby the Police Department would occupy the Tully Road Stables, housing the Mounted, Parks enforcement and Canine Units, in consideration for patrol and support services at several City parks (Coyote River Park, Almaden Lake Park, Alum Rock Park and Lake Cunningham Park). One police sergeant was assigned full-time to the Tully Road Stables to coordinate the renovation of the debilitated facility. Utilizing City workers (electrician, plumbers, painters, etc.) and inmates assigned to the Public Service Program, the remodeling was completed with little cost to the City.

In 1986, Mayor Tom McEnery and Chief of Police Joseph D. McNamara formed a blue-ribbon Task Force to raise money to purchase needed equipment for the new full-time Unit. Approximately $70,000 was collected by private citizens and used to purchase 14 horses, 13 saddles, blankets and associated leather gear, one marked pick-up truck and one six-horse trailer.

In July 1985, a police officer was selected as the Unit's horse trainer, to be responsible for selecting and training the horses, as well as training all Unit members. The first five Unit members graduated from the 14-week academy in December of 1986, and the second group of five officers graduated in April of 1987.