将360电影映射到球体/实现vtkmpegrader类

Question

底部编辑的版本

我有一个360相机的电影镜头，我想映射到一个球体上。场景保存为MP4文件中的等矩形投影。你知道吗

我发现this example解释了如何将图像映射到球体上，并且能够用玛雅维动画decorator，从MP4文件中读取连续的帧，并将纹理映射到球体上（使用mayavi示例部分截取的代码将numpy数组转换为tvtk.ImageData图像数据对象）。你知道吗

然而，我遇到了一个问题，使我怀疑，我可能是用这个完全错误的。。。奇怪的圆形人工制品：Movie frame on sphere

当我将链接示例与电影中的JPG快照结合使用时，投影看起来就像它应该的样子（上面的部分看起来只是错误的，但这是视频片段的一部分）： Image on sphere

代码如下：

'''
Altered version of Andras Deak's
on https://stackoverflow.com/questions/53074908/map-an-image-onto-a-sphere-and-plot-3d-trajectories

'''

import imageio
from mayavi import mlab
from tvtk.api import tvtk # python wrappers for the C++ vtk ecosystem
from tvtk.tools import visual
from tvtk.common import configure_input_data, is_old_pipeline

def image_from_array(ary):
    """ Create a VTK image object that references the data in ary.
        The array is either 2D or 3D with.  The last dimension
        is always the number of channels.  It is only tested
        with 3 (RGB) or 4 (RGBA) channel images.
        Note: This works no matter what the ary type is (accept
        probably complex...).  uint8 gives results that make since
        to me.  Int32 and Float types give colors that I am not
        so sure about.  Need to look into this...
    """

    sz = ary.shape
    dims = len(sz)
    # create the vtk image data
    img = tvtk.ImageData()

    if dims == 2:
        # 1D array of pixels.
        img.whole_extent = (0, sz[0]-1, 0, 0, 0, 0)
        img.dimensions = sz[0], 1, 1
        img.point_data.scalars = ary

    elif dims == 3:
        # 2D array of pixels.
        if is_old_pipeline():
            img.whole_extent = (0, sz[0]-1, 0, sz[1]-1, 0, 0)
        else:
            img.extent = (0, sz[0]-1, 0, sz[1]-1, 0, 0)
        img.dimensions = sz[0], sz[1], 1

        # create a 2d view of the array
        ary_2d = ary[:]
        ary_2d.shape = sz[0]*sz[1],sz[2]
        img.point_data.scalars = ary_2d

    else:
        raise ValueError("ary must be 3 dimensional.")

    return img



# create a figure window (and scene)
fig = mlab.figure(size=(600, 600))
visual.set_viewer(fig)

# load video
vid = imageio.get_reader('movie.mp4', 'ffmpeg')

# use a TexturedSphereSource, a.k.a. getting our hands dirty
R = 1
Nrad = 180

# create the sphere source with a given radius and angular resolution
sphere = tvtk.TexturedSphereSource(radius=R, theta_resolution=Nrad, phi_resolution=Nrad)

# assemble rest of the pipeline, assign texture
sphere_mapper = tvtk.PolyDataMapper(input_connection=sphere.output_port)
sphere_actor = tvtk.Actor(mapper=sphere_mapper)

@mlab.show
@mlab.animate(delay=50)
def auto_sphere():

    for i in range(1,600):

        image = vid.get_data(i)
        img = image_from_array(image)
        texture = tvtk.Texture(interpolate=1)
        configure_input_data(texture, img)

        sphere_actor.texture = texture
        fig.scene.add_actor(sphere_actor)

        yield


auto_sphere()

我对这个话题完全陌生。如何才能正确地做到这一点？你知道吗

---

编辑：

所以我想我找到了真正的问题。但我还不知道怎么解决。问题似乎出在读取MP4文件的方式上。在这个修改过的版本中（使用MP4中各个帧的jpeg文件），vtk管道和生成的渲染电影看起来是正确的：

from mayavi import mlab
from tvtk.api import tvtk # python wrappers for the C++ vtk ecosystem

# create a figure
fig = mlab.figure(size=(600, 600))

# sphere source
R = 1
Nrad = 180
sphere = tvtk.TexturedSphereSource(radius=R, theta_resolution=Nrad, phi_resolution=Nrad)
# sphere mapper
sphere_mapper = tvtk.PolyDataMapper(input_connection=sphere.output_port)
# actor
sphere_actor = tvtk.Actor(mapper=sphere_mapper)
# image reader
image = tvtk.JPEGReader()
image.file_name = 'testdata/frame001.jpg'
# texture
texture = tvtk.Texture(input_connection=image.output_port, interpolate=1)
sphere_actor.texture = texture

# add actor to scene
fig.scene.add_actor(sphere_actor)

@mlab.show
@mlab.animate(delay=50)
def auto_sphere():

    for i in range(2,101):
        num = str(i)

        filepath = 'testdata/frame%s.jpg' % num.zfill(3)

        image.file_name = filepath

        # render current texture
        fig.scene.render()

        yield

auto_sphere()

我想我现在的新问题是： 我能实现一个定制的vtkImageReader2类或类似的python类吗？它允许我读取mp4的连续帧？如果是这样的话，那么如何才能做到这一点呢？ 不幸的是，我找不到任何关于如何做到这一点的教程。你知道吗