# Flutter Filament Cross-platform, Physically-based rendering inside Flutter applications. Flutter plugin wrapping the Filament renderer https://github.com/google/filament. Powers the Polyvox and odd-io engines. # Sponsors Thank you to odd-io for sponsoring work on supporting Windows, raycasting, testing and documentation. # Overview ## Versioning Last tested on Flutter `3.15.0-15.2.pre`. This is on the Flutter beta channel, so run: ``` flutter channel beta flutter upgrade ``` ||Android|iOS|MacOS|Windows|Linux|WebGL |---|---|---|---|---|---|| |Filament|v1.43.1 (arm64/armeabi-v7a/x86/x86_64)|v1.43.1* (arm64)|v1.43.1 (arm64)|v1.32.4 (x86_64)|TODO**|TODO***| |Flutter||3.15.0-15.2.pre|3.15.0-15.2.pre|3.15.0-15.2.pre * iOS release build has a skybox bug so the debug versions are currently shipped on iOS ** (Waiting for https://github.com/google/filament/issues/7078 to be resolved before upgrading, not sure exactly when the bug was introduced but it was somewhere between v1.32.4 and v1.40.0) *** Texture widget not currently supported on web in Flutter. ## Features |Feature|Supported| |---|---| |glTF|Y| |glb|Y| # Basic Setup ## Clone flutter_filament This plugin is not yet published to pub.dev. To use in your project, simply clone the repository and pull the latest binaries from Git LFS: ``` cd $HOME git clone && cd flutter_filament git lfs pull ``` You *do not need to build Filament yourself*. The repository is bundled with all necessary headers/static libraries (`windows/lib`, `ios/lib`, `macos/lib` and `linux/lib`) and the Flutter plugin has been configured to link at build time. If you want to run the example project to check: ``` cd example && flutter run -d ``` ## Add dependency Add the plugin as a dependency in the pubspec.yaml for your application: ``` name: your_project description: Your project ... dependencies: flutter: sdk: flutter flutter_filament: path: ``` # Basic Usage See the `example` project for a complete sample of the below steps. ## Creating the viewport widget and controller To embed a viewport in your app, create an instance of `FilamentControllerFFI` somewhere in your app: e.g. ``` class MyApp extends StatelessWidget { final _filamentController = FilamentControllerFFI(); ... } ``` Constructing this object only load symbols from the native FFI library. (Note this is not (yet) a singleton, so ensure it is placed somewhere in the widget hierachy where it will not be garbage-collected until you no longer need a rendering canvas). Next, create an instance of `FilamentWidget` in the widget hierarchy where you want the rendering canvas to appear. This can be sized as large or as small as you want. Flutter widgets can be positioned above or below the `FilamentWidget`. ``` class MyApp extends StatelessWidget { final _filamentController = FilamentControllerFFI(); @override Widget build(BuildContext context) { return MaterialApp( color: Colors.white, home: Scaffold(backgroundColor: Colors.white, body: Stack(children:[ Container(color:Colors.green, height:100, width:100), Positioned.fill(top:100, left:100child:FilamentWidget(controller:_filamentController)), Positioned(right:0, bottom:0, child:Container(color:Colors.purple, height:100, width:100)) ]))); } } ``` When a `FilamentWidget` is added to the widget hierarchy: 1) on the first frame, by default a Container will be rendered with solid red. If you want to change this, pass a widget as the `initial` paramer to the `FilamentWidget` constructor. 2) on the second frame, `FilamentWidget` will retrieve its actual size and request the `FilamentController` to create: a) the backing textures needed to insert a `Texture` widget into b) a rendering thread c) a `FilamentViewer` and an `AssetManager`, which will allow you to load assets/cameras/lighting/etc via the `FilamentController` If this was successful, the viewport should turn from red to black. ### Rendering By default, the FilamentController will only render into the viewport by manually calling `render()` on the FilamentController. This is to avoid needlessly running a render loop when there is nothing to display. To automatically render at 60fps, call `setRendering(true)` on `FilamentController`. ### Assets To add a glTF asset to the scene, call `loadGlb()` on `FilamentController` with the Flutter asset path to your .glb file. For example, if your `pubspec.yaml` looks like this: ``` flutter: assets: - assets/models/bob.glb ``` Then you would call the following ``` var entity = _filamentController.loadGlb("assets/models/bob.glb"); ``` You can also pass a URI to indicate that the glTF file should be loaded from the filesystem: ``` var entity = _filamentController.loadGlb("file:///tmp/bob.glb"); ``` The returned value is an integer handle that be used to manipulate the asset (better referred to as the "entity") in the scene. E.g. to remove the asset: ``` _filamentController.removeAsset(entity); ``` ### Camera movement To enable mouse/swipe navigation through the scene, wrap the `FilamentWidget` inside a `FilamentGestureDetector`: ``` class MyApp extends StatelessWidget { final _filamentController = FilamentControllerFFI(); @override Widget build(BuildContext context) { return MaterialApp( color: Colors.white, home: Scaffold(backgroundColor: Colors.white, body: Stack(children:[ Container(color:Colors.green, height:100, width:100), Positioned.fill(top:100, left:100,child:FilamentGestureDetector( controller: _filamentController, child:FilamentWidget( controller:_filamentController ))), Positioned(right:0, bottom:0, child:Container(color:Colors.purple, height:100, width:100)) ]))); } } ``` ## Advanced Usage If you want to work with custom materials, you will need some (basic knowledge of the underlying Filament library)[https://google.github.io/filament/Materials.html#compilingmaterials]. Things to keep in mind: - You must compile materials with the correct version of Filament (see the table above). Keep in mind that versions may not be identical across platforms so you may need multiple uberz files for multiple platforms. e.g. the lit_opaque.uberz file has been created from a Filament build: ``` cd out/cmake-android-release-aarch64/libs/gltfio uberz -TSHADINGMODEL=lit -TBLENDING=opaque -o lit_opaque_43.uberz lit_opaque ``` (note that the number in the filename corresponds to the Material version, not the Filament version. Not every Filament version requires a new Material version). # Building Filament from source ``` git clone git@github.com:nmfisher/filament.git && cd filament ``` ## Windows To support embedding GPU textures in Flutter (rather than copying to a CPU pixel buffer on every frame), we need to build a slightly customized version of Filament that uses GLES on Windows (rather than the default, which uses OpenGL). Separately, we also force the Filament gltfio library to load assets via in-memory buffers, rather than the filesystem. This is simply a convenience so we don't have to use different logic for gltf resource loading across platforms. ``` git checkout flutter-filament-windows mkdir out && cd out ``` Building notes: On Android/iOS, we remove -fno-exceptions from CMakeLists.txt Project structure: - most shared code/headers under ios/src (because I still can't get podspec to build a target with symlinks or relative paths) - building on MacOS, we currently just delete the macos/include and macos/src directories and copy from iOS (for same reason), e.g. `rm -r ../macos/include && cp -R ../ios/include ../macos && rm -r ../macos/src && cp -R ../ios/src ../macos && pushd macos && pod update && popd && flutter run -d macos` - Android keeps a separate copy of ALL headers (because it's currently running a different version of Filament, earlier versions have some texture filtering issues) -- can't symlink either? --- IMPORTANT - current version only works on Flutter 3.15.0-15.2.pre / Filament v1.43.1 - Note also need to specifically build imageio/png/tinyexr - if release build, then need to comment out -fno-exceptions # Linux specific (Fedora 34) Building Filament: env LIBRARY_PATH=/usr/lib/gcc/x86_64-redhat-linux/11/ CC=clang CXX=clang++ CXX_FLAGS="-v" LD_FLAGS="-v" FILAMENT_REQUIRES_CXXABI=true ./build.sh -c release Running example project: export PKG_CONFIG_PATH=/usr/lib/pkgconfig/:/usr/lib64/pkgconfig/ CPLUS_INCLUDE_PATH=/usr/include/gtk-3.0/:/usr/include/pango-1.0/:/usr/include/harfbuzz:/usr/include/cairo/:/usr/include/gdk-pixbuf-2.0/:/usr/include/atk-1.0/ Web: EMCC_CFLAGS="-Wno-missing-field-initializers -Wno-deprecated-literal-operator -fPIC" ./build.sh -c -p webgl -i debug EMCC_CFLAGS="-I/Users/nickfisher/Documents/filament/libs/utils/include -I/Users/nickfisher/Documents/filament/libs/image/include -I/Users/nickfisher/Documents/filament/libs/math/include -I../../..//third_party/basisu/encoder/ -I../../..//third_party/libpng/ -I../../..//third_party/tinyexr/ -fPIC" emmake make ## Materials glTF assets The default - there is a simple material (unlit/opaque) used for background images. This is created by: ``` filament/out/release/filament/bin/matc -a opengl -a metal -o materials/image.filamat materials/image.mat filament/out/release/filament/bin/resgen -c -p image -x ios/include/material/ materials/image.filamat ``` # Known issues On Windows, loading a glTF (but NOT a glb) may crash due to a race condition between uploading resource data to GPU memory and being freed on the host side. This has been fixed in recent versions of Filament, but other bugs on Windows prevent upgrading. Only workaround is to load a .glb file.