Without beginning or end
Everything we know has limits. People are born, grow tired, occupy one place at a time, and eventually die. God is different. Moses speaks of God as existing before the mountains and before the earth itself, from everlasting to everlasting (Psalm 90:2). God did not begin, and nothing can bring him to an end.
That is part of what Christians mean by infinite. It does not mean God is merely very old or very large. He is the eternal Creator, not one more being inside the world he made. Isaiah says that the everlasting God does not faint or grow weary, and that no one can fully search out his understanding (Isaiah 40:28).
Before the mountains were brought forth, or ever thou hadst formed the earth and the world, even from everlasting to everlasting, thou art God.
Not contained by space
God is present everywhere, yet he is not spread thinly across the universe like a gas or a force. When Solomon dedicated the temple in Jerusalem, he recognised that even heaven and the highest heaven could not contain God, much less a building made by human hands (1 Kings 8:27).
David asks where he could go away from God's Spirit, then names the heights of heaven, the depths, the sea, and darkness. His point is not that there is nowhere private or safe from a harsh watcher. It is that God's presence reaches us in every place, including places where we feel lost or alone (Psalm 139:7-10).
Whither shall I go from thy spirit? or whither shall I flee from thy presence? If I ascend up into heaven, thou art there: if I make my bed in hell, behold, thou art there. If I take the wings of the morning, and dwell in the uttermost parts of the sea; Even there shall thy hand lead me, and thy right hand shall hold me.
Unlimited in wisdom and power
God's infinity also means his knowledge and strength have no boundary outside him. Job's friend Zophar asks whether anyone can find out the full measure of God, comparing God's greatness with heaven, the earth, and the sea (Job 11:7-9). The comparison reminds us that God will never fit neatly inside our explanations.
Still, infinite does not mean unknowable in every sense. We cannot comprehend God completely, but we can truly know what he has made known. Scripture points to his wisdom in creation and his care for those who come to him (Isaiah 40:28). A child can know their parent truly without knowing everything their parent knows. In a far greater way, people can know God while remaining humble before his greatness.
Infinite, yet near
God's greatness is not meant to make prayer feel pointless. Paul told the people of Athens that the God who made the world does not depend on human temples or gifts, because he gives life and breath to all. He also said God is not far from any one of us (Acts 17:24-25, Acts 17:27).
So God's infinity can be a comfort. Your needs do not exhaust him. Your questions do not overwhelm him. And the parts of life you cannot measure or understand are not beyond his sight. Read the passages slowly, and let both truths stand together: God is greater than we can grasp, and he is near enough to be sought.
That they should seek the Lord, if haply they might feel after him, and find him, though he be not far from every one of us: