The Desai House is situated in the village of Arabo at Dhargalim in the Pernem taluka. It is known to be built between 1885 and 1890 and is a reconstruction of an older house that was situated at the same location. It was purchased by the Desai family of Arabo, Dhargalim from a Muslim merchant who put forth a condition that the flagpole installed in the house should be venerated every Moharram festival.

Thus according to legend the Moharram festival in the village of Arabo was a festival celebrating the Hindu-Muslim unity wherein the Muslims from Arabo as well as Dhargalim would meet halfway and take a procession towards the river. They would be joined by the Hindus of both villages and would then immerse the floats prepared.

The Desai House has 24 rooms which are built around a central courtyard. Upstairs one can see large halls and twin towers set in exposed laterite that give a great view of the Chapora river. This stone structure is known to be built by Jaywantaji Roasaheb Jaiwantrao Desai.