Globally, 1.5 billion people live off the grid, with their only access to electricity often limited to operationally expensive fossil fuel generators. Solar power has risen as a sustainable and less costly option, but its generation is variable during the day and nonexistent at night. Thanks to recent technological advances, which have made large-scale electricity storage economically viable, a combination of solar generation and storage holds the promise of cheaper, greener, and more reliable off-grid power in the future. Still, it is not yet well understood how to jointly determine optimal capacity levels for renewable generation and storage. Our work aims to shed light on this question by developing a model of strategic capacity investment in both renewable generation and storage to match demand with supply in off-grid use cases while relying on fossil fuel as backup. Despite the complexity of the underlying model, we are able to extract two general results. First, we find that solar capacity and storage capacity are strategic complements, except in cases with very high investment in generation capacity, when they surprisingly turn into strategic substitutes with implications for long-term investment decisions. Second, we develop a simple heuristic to determine which storage technology, within a given portfolio, can turn a profit in the broadest set of market conditions and thus, is likely to be adopted first. We find that currently, low-efficiency, cheap technologies, such as thermal, can more easily turn a profit in off-grid applications than high-efficiency, expensive ones, such as lithium-ion batteries. We then develop two newsvendor-like approximations of the general model that are analytically tractable, yield precise values for the optimal investment decisions and profit in some cases, and provide bounds to the optimal investment decisions and profits in all other cases. To conclude, we calibrate our models to measure the accuracy of our solutions utilizing real-life data from three geographically-diverse islands, and then, we use our approximations to provide high-level insights on the role that large-scale storage will play in the years ahead as technology improves, carbon taxes are levied, and solar becomes cheaper.