As a group of task-agnostic pre-trained large-scale neural network models that can be later adapted to numerous downstream tasks, foundation models have made a significant impact on academia, industry, and society. Meanwhile, several efforts have been made to utilize and adapt foundation models for the geography and geoscience domain. In this presentation, we briefly discuss several recent works of Dr. Mai which utilize and adapt foundation models on various geospatial tasks such as location description extraction from disaster-related social media posts, sustainability index prediction, and image geolocalization.  We highlight that one of the uniqueness of geospatial tasks is they usually require locations as model input (e.g., sustainability index prediction) or generate locations (e.g., image geolocalization). However, all the current foundation models cannot handle geographic coordinates. To tackle this, we propose the idea of spatial representation learning (SRL) which aims at learning general-purpose neural network representations from various types of spatial data (e.g., points, polylines, polygons, networks, images, etc.) in their native formats. We propose TorchSpatial, a learning framework and benchmark for location (point) encoding, which is one of the most fundamental data types of spatial representation learning. We believe TorchSpatial will foster future advancement of spatial representation learning and geo-foundation model research. The TorchSpatial package is available at github.com/seai-lab/TorchSpatial.