Android Views in 3D

ViroCore also enables developers to enhance their applications with interactive 2D Android views by rendering them in AR or VR! As such, developers can easily build their familiar XML layouts, Android Buttons and ListViews in Android, and ViroCore will do all the heavy-lifting of rendering those views in a 3D space.

To use 2D Android UI components in VR:

1. Create an AndroidViewTexture with a width and height size representing your Android layout.
2. Attach your Android layout view to the AndroidViewTexture.
3. Apply this texture to any geometry you want to render the Android views on - a Cube, a Sphere or a Quad!

You can also refer to the example code below for a more detailed implementation walkthrough.

View Sizes and Hardware Support

When constructing an AndroidViewTexture in Virocore, it is important to consider view sizing limitations - that is, the area of your Android View layout that is rendered before it gets clipped.

By default, ViroCore's AndroidViewTexture will render AndroidViews onto a Viro Texture without any hardware acceleration. This ensures maximum compatibility with most Android OS versions, while guaranteeing correct view interaction behavior. However, there is a memory size limitation with this approach, resulting in potentially smaller viable width and height sizes for the texture.

To check if your desired view size works with software acceleration, you can also use the helper function AndroidViewTexture.suuportsSoftwareSurfaceOfSize(Context, desiredWidth, desiredHeight)

Else, developers can also force the AndroidViewTexture to use hardware acceleration, enabling larger texture sizes. Although most basic Android controls function properly with this mode, it is important to note that some views may have unpredictable behavior, particularly when handling user interaction.

🚧

Note: If you see a black texture instead of your expected Android views, this is most likely an indication that the set width and height pixel values on your AndroidViewTexture has exceeded the device's capability to render them.

Events and Interactivity

Users can also interact with and click on Android views that are attached to AndroidViewTextures in 3D (for example clicking on an Android Button with a VR reticle).

To achieve this behavior, developers can use the dispatchClickStateEvents API on an AndroidViewTexture to map a Viro click event to a pixel coordinate within the bounds of the AndroidViewTexture's size. With that API, ViroCore then effectively plumbs through those click events to be triggered normally as if it were and Android click event being passed onto your Android views. Note that the mapping of any click event's position to an AndroidViewTexture's rectangular layout will be determined by the geometry shape that you have applied that texture to (however we have made it easy to do so for flat rectangular surfaces, as shown below).

For convenience, developers can also use the AndroidViewtexture.getClickListenerWidhQuad(quad) helper function. This helper function helps construct a Viro ClickListener that maps Viro clicks performed on the given quad to Android touch events, which will then be automatically applied to the underlying view. This also takes into account the transformations that are applied to the quad, and as well as the size of the underlying texture.

Example Code:

In this example, we'll be attempting to render our Android layout shown below in AR. This layout will have an Android image, Android Text and an Android Button, all encapsulated within a RelativeLayout.

activity_main.xml

The representation of the above view in xml:

XML

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/activity_main"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:background="#99FFFFFF">

    <TextView
        android:id="@+id/custom_title"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_centerHorizontal="true"
        android:layout_centerVertical="true"
        android:text="Android Text: Hello World!" />

    <ImageView
        android:layout_width="200px"
        android:layout_height="200px"
        android:layout_centerHorizontal="true"
        android:src="@drawable/icons_admin"
        android:layout_above="@id/custom_title"/>

    <Button
        android:id="@+id/button1"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_centerHorizontal="true"
        android:layout_below="@id/custom_title"
        android:text="Android Button"
        android:clickable="true"/>
</RelativeLayout>

Next, we'll perform the usual Android view setup of inflating our views and adding click listeners to them. In the example below, we will be changing the text when the user clicks on the Android button.

Java

// Basic Android view setting (Create / bind your layouts here).
private View inflateAndroidView(){
    final View androidViews = getLayoutInflater().inflate(R.layout.activity_main, null);
    final TextView text = (TextView) androidViews.findViewById(R.id.custom_title);
    final Button button = (Button) androidViews.findViewById(R.id.button1);
    button.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            text.setText("Android Text: Clicked on Button!");
        }
    });
    
    return androidViews;
}

Finally, upon renderer startup, we'll create our AndroidViewTexture, and attach our desired android views to it. We then apply this texture onto a surface and set the appropriate click listener such that users can interact with this view.

Java

// Basic Android view setting (Create / bind your layouts here).
final View androidViews = inflateAndroidView();

// Create a Viro AndroidViewTexture that represents our Views.
int pxWidth = 400;
int pxHeight = 822;
boolean isAccelerated = true;
AndroidViewTexture androidTexture = new AndroidViewTexture(mViroView, pxWidth, pxHeight, isAccelerated);
androidTexture.attachView(androidViews);

// Set the Texture to be used on our surface in 3D.
final Material material = new Material();
material.setDiffuseTexture(androidTexture);

Quad surface = new Quad(0.5f, 1f);
surface.setMaterials(Arrays.asList(material));
Node surfaceNode = new Node();
surfaceNode.setGeometry(surface);
surfaceNode.setPosition(new Vector(0,0,-1));

// Add clicklisteners to 
surfaceNode.setClickListener(androidTexture.getClickListenerWithQuad(surface));

// Add the Surface to the scene.
mARScene.getRootNode().addChildNode(surfaceNode);

Final Result:

🚧

Known Limitations

AndroidViewTextures only supports rendering basic Android View Controls in AR/VR. However, rendering Android Views that contains GLSurfaces or dynamic web views are not supported.