The variety of exterior powers of linear maps

Let $K$ be a field and $V$ and $W$ be $K$-vector spaces of dimension $m$ and $n$. Let $\phi$ be the canonical map from $Hom(V,W)$ to $Hom(\wedge^t V,\wedge^t W)$. We investigate the Zariski closure $X_t$ of the image $Y_t$ of $\phi$. In the case $t=\min(m,n)$, $Y_t=X_t$ is the cone over a Grassmannian, but $X_t$ is larger than $Y_t$ for $1