A significant issue that contributes to worldwide water shortage is a mismatch between freshwater demand and supply. Exploration and use of rainwater seem to be feasible solutions for reducing the above-mentioned problem. The potential and difficulties of Malaysia's rainwater harvesting system (RWHS) are discussed in this paper. Malaysia may be classified as a nation with a lot of yearly rainfall and a lot of household water use. Malaysia is therefore in a good position to collect rainwater for both potable and non-potable purposes. Despite the fact that the RWH guidelines were published in Malaysia in 1999, the use of RWHS as an alternative water resource is still restricted owing to its lengthy payback period and low public acceptability. The cost of implementation, the wide application of commercial buildings, a cost-effective treatment system, and effective policy implementation, the use of green materials, public perception improvement, and reliable first flush technology are all major future challenges for RWHS implementation in Malaysia. Some suggestions, such as providing enough subsidies and restricting the use of piped water, are required for RWHS to be implemented at a larger scale.