Bootstrap 5 no longer needs jQuery so it's easier to use with Vue, and no longer requires a library like bootstrap-vue.

Install bootstrap as you would any other JS module in the Vue project using npm install or by adding it to the package.json...

npm install --save bootstrap
npm install --save @popperjs/core

Next, add the Bootstrap CSS and JS components to the Vue project entrypoint (ie: src/main.js)...

import "bootstrap/dist/css/bootstrap.min.css"
import "bootstrap"

Then, the simplest way to use Bootstrap components is via the data-bs- attributes. For example here's the Bootstrap Collapse component...

<button 
  class="btn btn-primary" 
  data-bs-target="#collapseTarget" 
  data-bs-toggle="collapse">
  Bootstrap collapse
</button>
<div class="collapse py-2" id="collapseTarget">
  This is the toggle-able content!
</div>

Demo with Navbar component

Or, you can import any Bootstrap components and "wrap" them as Vue components. For example here's the Popover component...

import { Popover } from bootstrap;

const popover = Vue.component('bsPopover', {
  template: `
    <slot/>
  `,
  props: {
    content: {
      required: false,
      default: '',
    },
    title: {
      default: 'My Popover',
    },
    trigger: {
      default: 'click',
    },
    delay: {
      default: 0,
    },
    html: {
      default: false,
    },
  },
  mounted() {
    // pass bootstrap popover options from props
    var options = this.$props
    var ele = this.$slots.default[0].elm
    new Popover(ele,options)
  },
})

<bs-popover
  title="Hello Popover"
  content="This is my content for the popover!"
  trigger="hover">
    <button class="btn btn-danger">
      Hover for popover
    </button>
</bs-popover>

Demo | Read more

Bootstrap Vue is not yet ready for Vue 3.

To answer part of your question, Vue 3 changes the method for instantiating the application instance, including how plugins are registered.

For example...

import { createApp } from 'vue';
import Router from './router/Router';

createApp({/* options */}})
  .use(Router)
  .mount('#app');

You can read more about this at the official docs.

https://v3.vuejs.org/guide/instance.html

https://v3-migration.vuejs.org