The correct understanding is, it is our personal desires to please and serve Krsna that keeps us in the spiritual world.

However, when those desires to please Krsna are directed to only please ourselves and not Kṛṣṇa (self centred desires), they then force us out of the spiritual worlds of the Vaikuṇṭha planets and Goloka-Vrindavana because Krsna is no longer in the centre.

The jiva-souls are perpetually endowed with "free will" expressed fully in the spiritual world only. 

Being "marginal" means the jiva-souls always have free will in the spiritual world's of the Vaikuṇṭha planets and Goloka Vrindavana, also meaning they can leave anytime they like because of that free will.

So, if we came to this material world due to our choice to reject Krsna, that does NOT mean we are desireless while in the spiritual world. No, desires are always there but centred around Krsna in the spiritual world, and centred around ourselves in the material world.

The foolish choice to leave Krsna's association means one wants to explore their own options in the material world, and this also means they eventually do NOT want to be in the material world either and try to end it all by becoming inactive or dormant in the impersonal brahmajyoti.

So, how could one say the jiva-soul chose the material world, when the jiva-soul didn’t even know the material world exists?

No, it is not like that, the jiva-souls originally ALWAYS have desires to please Krsna but those same desires are now self centred gratification without Krsna.

The real meaning of all this is one's choices and desires ALWAYS exist, but by rejecting Krsna, those SAME choices and desires become "self centred" and no longer Krsna centred.

It is that "original choice of rejecting Krsna" that takes the jiva-soul out of the spiritual world and eventually places them in the material creation under the jurisdiction of Maha-Visnu without even knowing what the material world is.

All they know is their desires were no longer directed to please Krsna anymore.

The jiva-souls ALWAYS have their own individual desires and choices of what they want to do to please Krsna or Visnu in the spiritual world, but now those same desires are no longer Krsna centred, they are self centred. 

"Desire" is always there in both the spiritual world  and the material world. 

This is how the jiva-soul chooses to reject Krsna, their desires simply change from Krsna centred to self centred.

So now separated from Krsna, they simple put themselves in the center instead of Krsna.

"Desires" are always there.

All the desires the jiva-souls had previously was to serve Kṛṣṇa, where such desires are eternal, but now being independent from Krsna, those same desires are self serving.

This means Krsna is no longer in the centre and is ignored.

When the jīva-souls choose to reject Krsna in the spiritual planets, they then attempt to imitate Krsna, that is when they are instantaneous placed in the material creation under the jurisdiction of Maha-Visnu's dreams, where all "desires" are still real, but temperory.

Every desire that now does not include Krsna, is already in "blue-print" within the dreams of Maha-Visnu.

In order for the rebellious jiva-souls to make a choice, they ALREADY previously knew what their options were because they had those same desires for Krsna before they rejected Krsna and put themselves in the middle.

The only difference is Krsna is no longer the object of one's loving effection, the jiva-souls NEVER beome desireless. 

In the spiritual world you cannot compare the "desires, choices and fee will" of the individual jiva-souls to the mundane material creation because while still in the spiritual worlds, the jiva-souls have no information about the material creation.

The spiritual world is always in the "eternal presence" of Krsna and NEVER influenced by the material creations past, present and future, that only exists in the temporary decaying material creation.

By existing under the jurisdiction of Krsna's expansion Maha-Visnu in the material world, what ever the jiva-souls desire, they can still be fulfilled by Maha-Visnu.

This is achieved when the jiva-souls enter one of His dreams of the material creation that are simultaneously in sync with their own desires.

Only in the permanent spiritual world do the marginal living entities (individual jiva-souls) fully express their full potential, and never in the temporary mundane material creation.

The individual jiva-souls are eternal PERSONS, AS an eternal spiritual bodily svarupa form, like Krsnas, and ALWAYS have individual desires (but suppressed temporarily if they foolishly enter the dormant (inactive) impersonal brahmajyoti after first falling to the material world)

However, sometimes, very rarely in the spiritual world, the jiva-soul's desires to reciprocate and please Krsna is replaced by self centred desires to now only please themselves. When this happens they can no longer stay in either the Vaikuntha planets or Goloka Vrindavana.

As said above, the jiva-souls can never become desireless, maybe for a short time by those impersonalists who enter the impersonal Brahmajyoti, but they will always fall down from there because the eternal nature of the individual jiva-souls are to be always active and ALWAYS have desires, but ultimately to please Krsna.

So, when some jiva-souls reject Krsna their desires to please Him remain, they are still there, but are now self centred.

The jiva-souls in their natural infinite position are never desireless, which is the goal of impersonalists who foolishly believe the jiva-souls originated from the impersonal inactive Brahmajyoti.

It is a nonsense philosophy to say that we didn’t come from Krsna's abode of Goloka Vrindavana, which is the jiva-souls (marginal living entities) REAL infinite eternal home. Krsna states twice in the Srimad Bhagavatam that we left Him.

Canto 4 Chapter 28 text 53 and text 55.

Krsna directly asks the jiva-soul,

"Do you remember Me!!" (How can you remember what you never knew?)

Krsna directly says 'hitvā māṁ,' you rejected me, He says this in Srimad Bhagavatam Canto 4 Chapter 28 text 55, sa tvaṁ vihāya māṁ bandho, "You are the one who rejected Me, O friend!" [sa tvaṁ vihāya māṁ bandho], yes, we should now be determined to return back home back to Godhead, to again be with Krsna.” (SB, Canto 4 Ch 28 text 55)

Srila Prabhupāda explains here from the 4th Canto of Srimad Bhagavatam that our original infinite home, from where we all have came to the temporary decaying material world from, is Goloka-Vrindavana or the Vaikuntha planets, the perpetual Kingdoms of God (Krsna).

Srimad Bhagavatam - "The brahmana continued, "My dear friend, even though you cannot immediately recognize Me, can’t you remember that in the past you had a very intimate friend? Unfortunately, you gave up My company and accepted a position as enjoyer of this material world." (SB, Canto 4 Ch 28 Text 53)