Blog

  • vuh

    Watch out!

    I do not have time or interest any more to further maintain or develop this. I will lazily review the bug-fix requests but no(t much) effort is made to maintain the code quality or style and no new releases will be made.

    Vuh. A Vulkan-based GPGPU computing framework.

    Build Status

    Vulkan is the most widely supported GPU programming API on modern hardware/OS. It allows to write truly portable and performant GPU accelerated code that would run on iOS, Android, Linux, Windows, macOS… NVidia, AMD, Intel, Adreno, Mali… whatever. At the price of ridiculous amount of boilerplate. Vuh aims to reduce the boilerplate to (a reasonable) minimum in most common GPGPU computing scenarios. The ultimate goal is to beat OpenCL in usability, portability and performance.

    Motivating Example

    saxpy implementation using vuh.

    auto main()-> int {
       auto y = std::vector<float>(128, 1.0f);
       auto x = std::vector<float>(128, 2.0f);
    
       auto instance = vuh::Instance();
       auto device = instance.devices().at(0);    // just get the first available device
    
       auto d_y = vuh::Array<float>(device, y);   // create device arrays and copy data
       auto d_x = vuh::Array<float>(device, x);
    
       using Specs = vuh::typelist<uint32_t>;     // shader specialization constants interface
       struct Params{uint32_t size; float a;};    // shader push-constants interface
       auto program = vuh::Program<Specs, Params>(device, "saxpy.spv"); // load shader
       program.grid(128/64).spec(64)({128, 0.1}, d_y, d_x); // run once, wait for completion
    
       d_y.toHost(begin(y));                      // copy data back to host
    
       return 0;
    }

    and the corresponding kernel (glsl compute shader) code:

    layout(local_size_x_id = 0) in;             // workgroup size (set with .spec(64) on C++ side)
    layout(push_constant) uniform Parameters {  // push constants (set with {128, 0.1} on C++ side)
       uint size;                               // array size
       float a;                                 // scaling parameter
    } params;
    
    layout(std430, binding = 0) buffer lay0 { float arr_y[]; }; // array parameters
    layout(std430, binding = 1) buffer lay1 { float arr_x[]; };
    
    void main(){
       const uint id = gl_GlobalInvocationID.x; // current offset
       if(params.size <= id){                   // drop threads outside the buffer
          return;
       }
       arr_y[id] += params.a*arr_x[id];         // saxpy
    }

    Features

    • storage buffers as vuh::Array<T>
      • allocated in device-local, host-visible or device-local-host-visible memories
      • data exchange with host incl. hidden staging buffers
    • computation kernels as vuh::Program
      • buffers binding (passing arbitrary number of array parameters)
      • specialization constants (to set workgroup dimensions, etc…)
      • push-constants (to pass small data (<= 128 Bytes), like task dimensions etc…)
      • whatever compute shaders support, shared memory, etc…
    • asynchronous data transfer and kernel execution with host-side synchronization
    • multiple device support
    • yet to come…
    • not ever coming…

    Usage

    Visit original content creator repository https://github.com/Glavnokoman/vuh
  • shared-query-builder

    Andante Project Logo

    Shared Query Builder

    Doctrine 2/3 Query Builder decorator – AndanteProject

    Latest Version Github actions Php8 PhpStan

    A Doctrine 2 Query Builder decorator that makes easier to build your query in shared contexts.

    Why do I need this?

    When your query business logic is big and complex you are probably going to split its building process to different places/classes.

    Without SharedQueryBuilder there is no way to do that unless guessing Entity aliases and messing up with join statements.

    This query builder decorator addresses some problems you can find in a real world situation you usually solve with workarounds and business conventions.

    Features

    • Ask query builder which alias is used for an entity when you are outside its creation context;
    • Lazy joins to declare join statements to be performed only if related criteria are defined;
    • Immutable and unique query parameters;
    • Works like magic ✨.

    Requirements

    Doctrine 2 and PHP 7.4.

    Install

    Via Composer:

    $ composer require andanteproject/shared-query-builder

    Set up

    After creating your query builder, wrap it inside our SharedQueryBuilder.

    use Andante\Doctrine\ORM\SharedQueryBuilder;
    
    // $qb instanceof Doctrine\ORM\QueryBuilder
    // $userRepository instanceof Doctrine\ORM\EntityRepository
    $qb = $userRepository->createQueryBuilder('u');
    // Let's wrap query builder inside our decorator.
    // We use $sqb as acronym of "Shared Query Builder"
    $sqb = SharedQueryBuilder::wrap($qb);

    From now on, you can use $sqb exactly as you usually do with query builder (every single method of QueryBuilder is available on SharedQueryBuilder), but with some useful extra methods 🤫.

    When you’re done building your query, just unwrap your SharedQueryBuilder.

    // $sqb instanceof Andante\Doctrine\ORM\SharedQueryBuilder
    // $qb instanceof Doctrine\ORM\QueryBuilder
    $qb = $sqb->unwrap();

    Please note:

    • The only condition applied to build a SharedQueryBuilder is that no join statement has to be declared yet.
    • SharedQueryBuilder is a decorator of QueryBuilder, which means it is not an instance of QueryBuilder even if it has all its methods (sadly, Doctrine has no QueryBuilder Interface 🥺).
    • SharedQueryBuilder do not allow you to join an Entity multiple times with different aliases.

    Which additional methods do I have?

    Entity methods

    You can ask the SharedQueryBuilder if it has and entity in the from statement or some join statements.

    if($sqb->hasEntity(User::class)) // bool returned 
    { 
        // Apply some query criteria only if this query builder is handling the User entity
    }

    You can ask which is the alias of an Entity inside the query you’re building (no matter if it is used in a from statement or a join statement).

    $userAlias = $sqb->getAliasForEntity(User::class); // string 'u' returned 

    You can use withAlias method to smoothly add a condition for that entity property:

    if($sqb->hasEntity(User::class)) // bool returned 
    { 
        $sqb
            ->andWhere(
                $sqb->expr()->eq(
                    $sqb->withAlias(User::class, 'email'), // string 'u.email'
                    ':email_value'
                )
            )
            ->setParameter('email_value', 'user@email.com')
        ;    
    } 

    Given an alias, you can retrieve its entity class:

    $entityClass = $sqb->getEntityForAlias('u'); // string 'App\Entity\User' returned

    QueryBuilder::getAllAliases is extended to have an optional bool argument $includeLazy (default:false) to include lazy joins aliases.

    $allAliases = $sqb->getAllAliases(true);

    Lazy Join

    All query builder join methods can be used as usually, but you can also use them with “lazy” prefix.

    // Common join methods
    $sqb->join(/* args */);
    $sqb->innerJoin(/* args */);
    $sqb->leftJoin(/* args */);
    
    // Lazy join methods
    $sqb->lazyJoin(/* args */);
    $sqb->lazyInnerJoin(/* args */);
    $sqb->lazyLeftJoin(/* args */);
    
    // They works with all the ways you know you can perform joins in Doctrine
    // A: $sqb->lazyJoin('u.address', 'a') 
    // or B: $sqb->lazyJoin('Address::class', 'a', Expr\Join::WITH, $sqb->expr()->eq('u.address','a')) 

    By doing this, you are defining a join statement without actually adding it to your DQL query. It is going to be added to your DQL query only when you add another condition/dql part which refers to it. Automagically ✨.

    Based on how much confused you are right now, you can check for why you should need this or some examples to achieve your “OMG” revelation moment.

    Examples

    Let’s suppose we need to list User entities but we also have an optional filter to search an user by it’s address Building name.

    There is no need to perform any join until we decide to use that filter. We can use Lazy Join to achieve this.

    $sqb = SharedQueryBuilder::wrap($userRepository->createQueryBuilder('u'));
    $sqb
        ->lazyJoin('u.address', 'a')
        ->lazyJoin('a.building', 'b')
        //Let's add a WHERE condition that do not need our lazy joins 
        ->andWhere(
            $sqb->expr()->eq('u.verifiedEmail', ':verified_email')
        )
        ->setParameter('verified_email', true)
    ;
    
    $users = $sqb->getQuery()->getResults();
    // DQL executed:
    //     SELECT u
    //     FROM App\entity\User
    //     WHERE u.verifiedEmail = true
    
    // BUT if we use the same Query Builder to filter by building.name:
    $buildingNameFilter = 'Building A';
    $sqb
        ->andWhere(
            $sqb->expr()->eq('b.name', ':name_value')
        )
        ->setParameter('name_value', $buildingNameFilter)
    ;
    $users = $sqb->getQuery()->getResults();
    // DQL executed:
    //     SELECT u
    //     FROM App\entity\User
    //       JOIN u.address a
    //       JOIN a.building b
    //     WHERE u.verifiedEmail = true
    //       AND b.name = 'Building A'

    You are probably thinking: why don’t we achieve the same result with the following, more common, way? (keep in mind that avoid to perform unecessary joins is still a requirement)

    // How you could achieve this without SharedQueryBuilder
    $buildingNameFilter = 'Building A';
    $qb = $userRepository->createQueryBuilder('u');
    $qb
        ->andWhere(
            $qb->expr()->eq('u.verifiedEmail', ':verified_email')
        )
        ->setParameter('verified_email', true);
        
    if(!empty($buildingNameFilter)){
        $qb
            ->lazyJoin('u.address', 'a')
            ->lazyJoin('a.building', 'b')
            ->andWhere(
                $qb->expr()->eq('b.name', ':building_name_value')
            )
            ->setParameter('building_name_value', $buildingNameFilter)
        ;
    }
    
    $users = $qb->getQuery()->getResults(); // Same result as example shown before
    // But this has some down sides further explained

    The code above is perfectly fine if you build this whole query in the same context:

    • 👍 You are aware of the whole query building process;
    • 👍 You are aware of which entities are involved;
    • 👍 You are aware of which alias are defined for each entity.
    • 👍 You are aware of which query parameters are defined and their purpose.

    But you have problems:

    • 👎 You are mixing query structure definition with optional filtering criteria.
    • 👎 Code is is quickly going to be an unreadable mess.

    A real world case

    If your query structure grows with lots of joins and filtering criteria, you are probably going to split all that business logic in different classes.

    For instance, in a backoffice Users list, you are probably going to define your main query to list entities in your controller and handle optional filters in some other classes.

    // UserController.php
    class UserController extends Controller
    {
        public function index(Request $request, UserRepository $userRepository) : Response
        {
            $qb = $userRepository->createQueryBuilder('u');
            $qb
                ->andWhere(
                    $qb->expr()->eq('u.verifiedEmail', ':verified_email')
                )
                ->setParameter('verified_email', true);
            
            // Now Apply some optional filters from Request
            // Let's suppose we have an "applyFilters" method which is giving QueryBuilder and Request
            // to and array of classes responsable to take care of filtering query results.  
            $this->applyFilters($qb, $request);
            
            // Maybe have some pagination logic here too. Check KnpLabs/knp-components which is perfect for this.
            
            $users = $qb->getQuery()->getResults();
            // Build our response with User entities list.
        }
    }

    Filter classes may look like this:

    // BuildingNameFilter.php
    class BuildingNameFilter implements FilterInterface
    {
        public function filter(QueryBuilder $qb, Request $request): void
        {
            $buildingNameFilter = $request->query->get('building-name');
            if(!empty($buildingNameFilter)){
                $qb
                    ->join('u.address', 'a')
                    ->join('a.building', 'b')
                    ->andWhere(
                        $qb->expr()->eq('b.name', ':building_name_value')
                    )
                    ->setParameter('building_name_value', $buildingNameFilter)
                ;
            }
        }
    }

    We are committing some multiple sins here! 💀 The context is changed.

    • 👎 You are not aware of the whole query building process. Is the given QueryBuilder even a query on User entity?;
    • 👎 You are not aware of which entities are involved. Which entities are already been joined?;
    • 👎 You are not aware of which aliases are defined for each entity. No way we are calling u.address by convention 🤨;
    • 👎 You are aware of what parameters have been defined ($qb->getParameters()), but you are not aware why they have been defined, for which purpose and you can also override them changing elsewhere behavior;
    • 👎 Our job in this context is just to apply some filter. We can change the query by adding some join statements but we should avoid that. What if another filter also need to perform those joins? Devastating. 😵

    This’s why SharedQueryBuilder is going to save your ass in these situations

    Let’s see how we can solve all these problems with SharedQueryBuilder (you can now guess why it is named like this).

    Using SharedQueryBuilder you can:

    • 👍 Define lazy join to allow them to be performed only if they are needed;
    • 👍 Define some parameters immutable to be sure value is not going to be changed elsewhere;
    • 👍 You can check if an entity is involved in a query and then apply some business logic;
    • 👍 You can ask the query builder which alias is used for a specific entity so you are not going to guess aliases or sharing them between classes using constants (I know you thought of that 🧐).
    // UserController.php
    use Andante\Doctrine\ORM\SharedQueryBuilder;
    
    class UserController extends Controller
    {
        public function index(Request $request, UserRepository $userRepository) : Response
        {
            $sqb = SharedQueryBuilder::wrap($userRepository->createQueryBuilder('u'));
            $sqb
                // Please note: Sure, you can mix "normal" join methods and "lazy" join methods
                ->lazyJoin('u.address', 'a')
                ->lazyJoin('a.building', 'b')
                ->andWhere($sqb->expr()->eq('u.verifiedEmail', ':verified_email'))
                ->setImmutableParameter('verified_email', true);
            
            // Now Apply some optional filters from Request
            // Let's suppose we have an "applyFilters" method which is giving QueryBuilder and Request
            // to and array of classes responsable to take care of filtering query results.  
            $this->applyFilters($sqb, $request);
            
            // Maybe have some pagination logic here too.
            // You probably need to unwrap the Query Builder now for this
            $qb = $sqb->unwrap();
            
            $users = $qb->getQuery()->getResults();
            // Build our response with User entities list.
        }
    }

    Filter classes will look like this:

    // BuildingNameFilter.php
    use Andante\Doctrine\ORM\SharedQueryBuilder;
    
    class BuildingNameFilter implements FilterInterface
    {
        public function filter(SharedQueryBuilder $sqb, Request $request): void
        {
            $buildingNameFilter = $request->query->get('building-name');
            // Let's check if Query has a Building entity in from or join DQL parts 🙌
            if($sqb->hasEntity(Building::class) && !empty($buildingNameFilter)){
                $sqb
                    ->andWhere(
                        // We can ask Query builder for the "Building" alias instead of guessing it/retrieve somewhere else 💋
                        $sqb->expr()->eq($sqb->withAlias(Building::class, 'name'), ':building_name_value')
                        // You can also use $sqb->getAliasForEntity(Building::class) to discover alias is 'b';
                    )
                    ->setImmutableParameter('building_name_value', $buildingNameFilter)
                ;
            }
        }
    }
    • 👍 No extra join statements executed when there is no need for them;
    • 👍 No way to change/override parameters value once defined;
    • 👍 We can discover if the Query Builder is handling an Entity and then apply our business logic;
    • 👍 We are not guessing entity aliases;
    • 👍 Our filter class is only responsible for filtering;
    • 👍 There can be multiple filter class handling different criteria on the same entity without having duplicated join statements;

    Immutable Parameters

    Shared query builder has Immutable Parameters. Once defined, they cannot be changed otherwise and Exception will be raised.

    // $sqb instanceof Andante\Doctrine\ORM\SharedQueryBuilder
    
    // set a common Query Builder parameter, as you are used to 
    $sqb->setParameter('parameter_name', 'parameterValue');
    
    // set an immutable common Query Builder parameter. It cannot be changed otherwise an exception will be raised.
    $sqb->setImmutableParameter('immutable_parameter_name', 'parameterValue');
    
    // get a collection of all query parameters (commons + immutables!)
    $sqb->getParameters();
    
    // get a collection of all immutable query parameters (exclude commons)
    $sqb->getImmutableParameters();
    
    // Sets a parameter and return parameter name as string instead of $sqb.
    $sqb->withParameter(':parameter_name', 'parameterValue');
    $sqb->withImmutableParameter(':immutable_parameter_name', 'parameterValue');
    // This allows you to write something like this:
    $sqb->expr()->eq('building.name', $sqb->withParameter(':building_name_value', $buildingNameFilter));
    
    // The two following methods sets "unique" parameters. See "Unique parameters" doc section for more...
    $sqb->withUniqueParameter(':parameter_name', 'parameterValue');
    $sqb->withUniqueImmutableParameter(':parameter_name', 'parameterValue');

    Set parameter and use it in expression at the same moment

    If you are sure you are not going to use a parameter in multiple places inside your query, you can write the following code 🙌

    $sqb
        ->andWhere(
            $sqb->expr()->eq(
                $sqb->withAlias(Building::class, 'name'), 
                ':building_name_value'
            )
        )
        ->setImmutableParameter('building_name_value', $buildingNameFilter)
    ;

    this way 👇👇👇

    $sqb
        ->andWhere(
            $sqb->expr()->eq(
                $sqb->withAlias(Building::class, 'name'), 
                $sqb->withImmutableParameter(':building_name_value', $buildingNameFilter) // return ":building_name_value" but also sets immutable parameter
            )
        )
    ;

    Unique parameters

    Beside immutable parameters, you can also demand query builder the generation of a parameter name. Using the following methods, query builder will decorate names to avoid conflicts with already declared ones ( which cannot even happen with immutable parameters).

    $sqb
        ->andWhere(
            $sqb->expr()->eq(
               'building.name', 
                $sqb->withUniqueParameter(':name', $buildingNameFilter) // return ":param_name_4b3403665fea6" making sure parameter name is not already in use and sets parameter value.
            )
        )
        ->andWhere(
            $sqb->expr()->gte(
               'building.createdAt', 
                $sqb->withUniqueImmutableParameter(':created_at', new \DateTime('-5 days ago'))  // return ":param_created_at_5819f3ad1c0ce" making sure parameter name is not already in use and sets immutable parameter value.
            )
        )
        ->andWhere(
            $sqb->expr()->lte(
               'building.createdAt',
                $sqb->withUniqueImmutableParameter(':created_at', new \DateTime('today midnight'))  // return ":param_created_at_604a8362bf00c" making sure parameter name is not already in use and sets immutable parameter value.
            )
        )
    ;
    
    /* 
     * Query Builder has now 3 parameters:
     *  - param_name_4b3403665fea6 (common)
     *  - param_created_at_5819f3ad1c0ce (immutable)
     *  - param_created_at_604a8362bf00c (immutable)
     */

    Conclusion

    The world is a happier place 💁.

    Give us a ⭐️ if your world is now a happier place too! 💃🏻

    Built with love ❤️ by AndanteProject team.

    Visit original content creator repository https://github.com/andanteproject/shared-query-builder
  • shared-query-builder

    Andante Project Logo

    Shared Query Builder

    Doctrine 2/3 Query Builder decorator – AndanteProject

    Latest Version Github actions Php8 PhpStan

    A Doctrine 2 Query Builder decorator that makes easier to build your query in shared contexts.

    Why do I need this?

    When your query business logic is big and complex you are probably going to split its building process to different places/classes.

    Without SharedQueryBuilder there is no way to do that unless guessing Entity aliases and messing up with join statements.

    This query builder decorator addresses some problems you can find in a real world situation you usually solve with workarounds and business conventions.

    Features

    • Ask query builder which alias is used for an entity when you are outside its creation context;
    • Lazy joins to declare join statements to be performed only if related criteria are defined;
    • Immutable and unique query parameters;
    • Works like magic ✨.

    Requirements

    Doctrine 2 and PHP 7.4.

    Install

    Via Composer:

    $ composer require andanteproject/shared-query-builder

    Set up

    After creating your query builder, wrap it inside our SharedQueryBuilder.

    use Andante\Doctrine\ORM\SharedQueryBuilder;
    
    // $qb instanceof Doctrine\ORM\QueryBuilder
    // $userRepository instanceof Doctrine\ORM\EntityRepository
    $qb = $userRepository->createQueryBuilder('u');
    // Let's wrap query builder inside our decorator.
    // We use $sqb as acronym of "Shared Query Builder"
    $sqb = SharedQueryBuilder::wrap($qb);

    From now on, you can use $sqb exactly as you usually do with query builder (every single method of QueryBuilder is available on SharedQueryBuilder), but with some useful extra methods 🤫.

    When you’re done building your query, just unwrap your SharedQueryBuilder.

    // $sqb instanceof Andante\Doctrine\ORM\SharedQueryBuilder
    // $qb instanceof Doctrine\ORM\QueryBuilder
    $qb = $sqb->unwrap();

    Please note:

    • The only condition applied to build a SharedQueryBuilder is that no join statement has to be declared yet.
    • SharedQueryBuilder is a decorator of QueryBuilder, which means it is not an instance of QueryBuilder even if it has all its methods (sadly, Doctrine has no QueryBuilder Interface 🥺).
    • SharedQueryBuilder do not allow you to join an Entity multiple times with different aliases.

    Which additional methods do I have?

    Entity methods

    You can ask the SharedQueryBuilder if it has and entity in the from statement or some join statements.

    if($sqb->hasEntity(User::class)) // bool returned 
    { 
        // Apply some query criteria only if this query builder is handling the User entity
    }

    You can ask which is the alias of an Entity inside the query you’re building (no matter if it is used in a from statement or a join statement).

    $userAlias = $sqb->getAliasForEntity(User::class); // string 'u' returned 

    You can use withAlias method to smoothly add a condition for that entity property:

    if($sqb->hasEntity(User::class)) // bool returned 
    { 
        $sqb
            ->andWhere(
                $sqb->expr()->eq(
                    $sqb->withAlias(User::class, 'email'), // string 'u.email'
                    ':email_value'
                )
            )
            ->setParameter('email_value', 'user@email.com')
        ;    
    } 

    Given an alias, you can retrieve its entity class:

    $entityClass = $sqb->getEntityForAlias('u'); // string 'App\Entity\User' returned

    QueryBuilder::getAllAliases is extended to have an optional bool argument $includeLazy (default:false) to include lazy joins aliases.

    $allAliases = $sqb->getAllAliases(true);

    Lazy Join

    All query builder join methods can be used as usually, but you can also use them with “lazy” prefix.

    // Common join methods
    $sqb->join(/* args */);
    $sqb->innerJoin(/* args */);
    $sqb->leftJoin(/* args */);
    
    // Lazy join methods
    $sqb->lazyJoin(/* args */);
    $sqb->lazyInnerJoin(/* args */);
    $sqb->lazyLeftJoin(/* args */);
    
    // They works with all the ways you know you can perform joins in Doctrine
    // A: $sqb->lazyJoin('u.address', 'a') 
    // or B: $sqb->lazyJoin('Address::class', 'a', Expr\Join::WITH, $sqb->expr()->eq('u.address','a')) 

    By doing this, you are defining a join statement without actually adding it to your DQL query. It is going to be added to your DQL query only when you add another condition/dql part which refers to it. Automagically ✨.

    Based on how much confused you are right now, you can check for why you should need this or some examples to achieve your “OMG” revelation moment.

    Examples

    Let’s suppose we need to list User entities but we also have an optional filter to search an user by it’s address Building name.

    There is no need to perform any join until we decide to use that filter. We can use Lazy Join to achieve this.

    $sqb = SharedQueryBuilder::wrap($userRepository->createQueryBuilder('u'));
    $sqb
        ->lazyJoin('u.address', 'a')
        ->lazyJoin('a.building', 'b')
        //Let's add a WHERE condition that do not need our lazy joins 
        ->andWhere(
            $sqb->expr()->eq('u.verifiedEmail', ':verified_email')
        )
        ->setParameter('verified_email', true)
    ;
    
    $users = $sqb->getQuery()->getResults();
    // DQL executed:
    //     SELECT u
    //     FROM App\entity\User
    //     WHERE u.verifiedEmail = true
    
    // BUT if we use the same Query Builder to filter by building.name:
    $buildingNameFilter = 'Building A';
    $sqb
        ->andWhere(
            $sqb->expr()->eq('b.name', ':name_value')
        )
        ->setParameter('name_value', $buildingNameFilter)
    ;
    $users = $sqb->getQuery()->getResults();
    // DQL executed:
    //     SELECT u
    //     FROM App\entity\User
    //       JOIN u.address a
    //       JOIN a.building b
    //     WHERE u.verifiedEmail = true
    //       AND b.name = 'Building A'

    You are probably thinking: why don’t we achieve the same result with the following, more common, way? (keep in mind that avoid to perform unecessary joins is still a requirement)

    // How you could achieve this without SharedQueryBuilder
    $buildingNameFilter = 'Building A';
    $qb = $userRepository->createQueryBuilder('u');
    $qb
        ->andWhere(
            $qb->expr()->eq('u.verifiedEmail', ':verified_email')
        )
        ->setParameter('verified_email', true);
        
    if(!empty($buildingNameFilter)){
        $qb
            ->lazyJoin('u.address', 'a')
            ->lazyJoin('a.building', 'b')
            ->andWhere(
                $qb->expr()->eq('b.name', ':building_name_value')
            )
            ->setParameter('building_name_value', $buildingNameFilter)
        ;
    }
    
    $users = $qb->getQuery()->getResults(); // Same result as example shown before
    // But this has some down sides further explained

    The code above is perfectly fine if you build this whole query in the same context:

    • 👍 You are aware of the whole query building process;
    • 👍 You are aware of which entities are involved;
    • 👍 You are aware of which alias are defined for each entity.
    • 👍 You are aware of which query parameters are defined and their purpose.

    But you have problems:

    • 👎 You are mixing query structure definition with optional filtering criteria.
    • 👎 Code is is quickly going to be an unreadable mess.

    A real world case

    If your query structure grows with lots of joins and filtering criteria, you are probably going to split all that business logic in different classes.

    For instance, in a backoffice Users list, you are probably going to define your main query to list entities in your controller and handle optional filters in some other classes.

    // UserController.php
    class UserController extends Controller
    {
        public function index(Request $request, UserRepository $userRepository) : Response
        {
            $qb = $userRepository->createQueryBuilder('u');
            $qb
                ->andWhere(
                    $qb->expr()->eq('u.verifiedEmail', ':verified_email')
                )
                ->setParameter('verified_email', true);
            
            // Now Apply some optional filters from Request
            // Let's suppose we have an "applyFilters" method which is giving QueryBuilder and Request
            // to and array of classes responsable to take care of filtering query results.  
            $this->applyFilters($qb, $request);
            
            // Maybe have some pagination logic here too. Check KnpLabs/knp-components which is perfect for this.
            
            $users = $qb->getQuery()->getResults();
            // Build our response with User entities list.
        }
    }

    Filter classes may look like this:

    // BuildingNameFilter.php
    class BuildingNameFilter implements FilterInterface
    {
        public function filter(QueryBuilder $qb, Request $request): void
        {
            $buildingNameFilter = $request->query->get('building-name');
            if(!empty($buildingNameFilter)){
                $qb
                    ->join('u.address', 'a')
                    ->join('a.building', 'b')
                    ->andWhere(
                        $qb->expr()->eq('b.name', ':building_name_value')
                    )
                    ->setParameter('building_name_value', $buildingNameFilter)
                ;
            }
        }
    }

    We are committing some multiple sins here! 💀 The context is changed.

    • 👎 You are not aware of the whole query building process. Is the given QueryBuilder even a query on User entity?;
    • 👎 You are not aware of which entities are involved. Which entities are already been joined?;
    • 👎 You are not aware of which aliases are defined for each entity. No way we are calling u.address by convention 🤨;
    • 👎 You are aware of what parameters have been defined ($qb->getParameters()), but you are not aware why they have been defined, for which purpose and you can also override them changing elsewhere behavior;
    • 👎 Our job in this context is just to apply some filter. We can change the query by adding some join statements but we should avoid that. What if another filter also need to perform those joins? Devastating. 😵

    This’s why SharedQueryBuilder is going to save your ass in these situations

    Let’s see how we can solve all these problems with SharedQueryBuilder (you can now guess why it is named like this).

    Using SharedQueryBuilder you can:

    • 👍 Define lazy join to allow them to be performed only if they are needed;
    • 👍 Define some parameters immutable to be sure value is not going to be changed elsewhere;
    • 👍 You can check if an entity is involved in a query and then apply some business logic;
    • 👍 You can ask the query builder which alias is used for a specific entity so you are not going to guess aliases or sharing them between classes using constants (I know you thought of that 🧐).
    // UserController.php
    use Andante\Doctrine\ORM\SharedQueryBuilder;
    
    class UserController extends Controller
    {
        public function index(Request $request, UserRepository $userRepository) : Response
        {
            $sqb = SharedQueryBuilder::wrap($userRepository->createQueryBuilder('u'));
            $sqb
                // Please note: Sure, you can mix "normal" join methods and "lazy" join methods
                ->lazyJoin('u.address', 'a')
                ->lazyJoin('a.building', 'b')
                ->andWhere($sqb->expr()->eq('u.verifiedEmail', ':verified_email'))
                ->setImmutableParameter('verified_email', true);
            
            // Now Apply some optional filters from Request
            // Let's suppose we have an "applyFilters" method which is giving QueryBuilder and Request
            // to and array of classes responsable to take care of filtering query results.  
            $this->applyFilters($sqb, $request);
            
            // Maybe have some pagination logic here too.
            // You probably need to unwrap the Query Builder now for this
            $qb = $sqb->unwrap();
            
            $users = $qb->getQuery()->getResults();
            // Build our response with User entities list.
        }
    }

    Filter classes will look like this:

    // BuildingNameFilter.php
    use Andante\Doctrine\ORM\SharedQueryBuilder;
    
    class BuildingNameFilter implements FilterInterface
    {
        public function filter(SharedQueryBuilder $sqb, Request $request): void
        {
            $buildingNameFilter = $request->query->get('building-name');
            // Let's check if Query has a Building entity in from or join DQL parts 🙌
            if($sqb->hasEntity(Building::class) && !empty($buildingNameFilter)){
                $sqb
                    ->andWhere(
                        // We can ask Query builder for the "Building" alias instead of guessing it/retrieve somewhere else 💋
                        $sqb->expr()->eq($sqb->withAlias(Building::class, 'name'), ':building_name_value')
                        // You can also use $sqb->getAliasForEntity(Building::class) to discover alias is 'b';
                    )
                    ->setImmutableParameter('building_name_value', $buildingNameFilter)
                ;
            }
        }
    }
    • 👍 No extra join statements executed when there is no need for them;
    • 👍 No way to change/override parameters value once defined;
    • 👍 We can discover if the Query Builder is handling an Entity and then apply our business logic;
    • 👍 We are not guessing entity aliases;
    • 👍 Our filter class is only responsible for filtering;
    • 👍 There can be multiple filter class handling different criteria on the same entity without having duplicated join statements;

    Immutable Parameters

    Shared query builder has Immutable Parameters. Once defined, they cannot be changed otherwise and Exception will be raised.

    // $sqb instanceof Andante\Doctrine\ORM\SharedQueryBuilder
    
    // set a common Query Builder parameter, as you are used to 
    $sqb->setParameter('parameter_name', 'parameterValue');
    
    // set an immutable common Query Builder parameter. It cannot be changed otherwise an exception will be raised.
    $sqb->setImmutableParameter('immutable_parameter_name', 'parameterValue');
    
    // get a collection of all query parameters (commons + immutables!)
    $sqb->getParameters();
    
    // get a collection of all immutable query parameters (exclude commons)
    $sqb->getImmutableParameters();
    
    // Sets a parameter and return parameter name as string instead of $sqb.
    $sqb->withParameter(':parameter_name', 'parameterValue');
    $sqb->withImmutableParameter(':immutable_parameter_name', 'parameterValue');
    // This allows you to write something like this:
    $sqb->expr()->eq('building.name', $sqb->withParameter(':building_name_value', $buildingNameFilter));
    
    // The two following methods sets "unique" parameters. See "Unique parameters" doc section for more...
    $sqb->withUniqueParameter(':parameter_name', 'parameterValue');
    $sqb->withUniqueImmutableParameter(':parameter_name', 'parameterValue');

    Set parameter and use it in expression at the same moment

    If you are sure you are not going to use a parameter in multiple places inside your query, you can write the following code 🙌

    $sqb
        ->andWhere(
            $sqb->expr()->eq(
                $sqb->withAlias(Building::class, 'name'), 
                ':building_name_value'
            )
        )
        ->setImmutableParameter('building_name_value', $buildingNameFilter)
    ;

    this way 👇👇👇

    $sqb
        ->andWhere(
            $sqb->expr()->eq(
                $sqb->withAlias(Building::class, 'name'), 
                $sqb->withImmutableParameter(':building_name_value', $buildingNameFilter) // return ":building_name_value" but also sets immutable parameter
            )
        )
    ;

    Unique parameters

    Beside immutable parameters, you can also demand query builder the generation of a parameter name. Using the following methods, query builder will decorate names to avoid conflicts with already declared ones ( which cannot even happen with immutable parameters).

    $sqb
        ->andWhere(
            $sqb->expr()->eq(
               'building.name', 
                $sqb->withUniqueParameter(':name', $buildingNameFilter) // return ":param_name_4b3403665fea6" making sure parameter name is not already in use and sets parameter value.
            )
        )
        ->andWhere(
            $sqb->expr()->gte(
               'building.createdAt', 
                $sqb->withUniqueImmutableParameter(':created_at', new \DateTime('-5 days ago'))  // return ":param_created_at_5819f3ad1c0ce" making sure parameter name is not already in use and sets immutable parameter value.
            )
        )
        ->andWhere(
            $sqb->expr()->lte(
               'building.createdAt',
                $sqb->withUniqueImmutableParameter(':created_at', new \DateTime('today midnight'))  // return ":param_created_at_604a8362bf00c" making sure parameter name is not already in use and sets immutable parameter value.
            )
        )
    ;
    
    /* 
     * Query Builder has now 3 parameters:
     *  - param_name_4b3403665fea6 (common)
     *  - param_created_at_5819f3ad1c0ce (immutable)
     *  - param_created_at_604a8362bf00c (immutable)
     */

    Conclusion

    The world is a happier place 💁.

    Give us a ⭐️ if your world is now a happier place too! 💃🏻

    Built with love ❤️ by AndanteProject team.

    Visit original content creator repository https://github.com/andanteproject/shared-query-builder
  • steel

    steel

    2d game framework pet project using C++17, SDL2, Entt and Catch2. Definitely not ready for serious
    usage.

    Inspiration

    Similarities are not coincidences, I’ve been taking a lot of inspiration from:

    Building the environment

    This project contains submodules, either clone it using

    git clone --recurse-submodules <repository_link>

    or, after cloning it, run:

    git submodule update --init --recursive

    and make sure that submodules under cmake and third_party were properly cloned.

    This project uses Miniconda as a package manager
    and conda devenv. After installing Miniconda you need
    to install conda-devenv on your base (root environment) with:

    conda activate base 
    conda install conda-devenv
    conda deactivate

    Then in order to create the environment run on the root of this project:

    conda devenv

    And to activate it, just call:

    conda activate steel

    Building the project

    On the root of this repository, execute:

    build_steel.bat

    Notes:

    1. Inside build/ there will be a Visual Studio Solution (steel.sln) and in order to make sure Visual Studio
      is initialized with all the correct environment variables, it’s a good idea to open it from the command line with
      steel environment active. I use an alias to make that easier:
    alias vs="C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\devenv.exe"

    Visit original content creator repository
    https://github.com/gcardozo123/steel

  • theme_comparator

    WordPress Theme Comparison Script

    Overview

    This Python script is designed to help developers and designers compare two versions of a WordPress theme. It identifies differences in files, including missing, added, and modified files, and provides a detailed report on the changes. This tool is invaluable for debugging, collaboration, and understanding changes between theme updates.

    Features

    • Extracts ZIP files: Automatically extracts WordPress theme ZIP files for comparison.
    • File Comparison: Compares each file in the two themes to identify missing, added, and modified files.
    • Detailed Diff Reports: Generates a unified diff report for modified files, showing precise changes at the line level.
    • Clear Output: Saves the comparison report with a timestamp and the names of the compared themes for easy reference.

    Use Cases

    • Theme Updates: Quickly see what has changed between two versions of a theme.
    • Debugging: Identify changes that may have introduced bugs or issues.
    • Collaboration: Understand what modifications have been made by team members.
    • Optimization: Track changes that affect performance or security.

    Requirements

    • Python 3.x
    • Standard Python libraries: zipfile, difflib, os, pathlib

    Installation

    1. Clone this repository to your local machine.
    2. Navigate to the project directory.

    Usage

    1. Setup: Create a directory with the following structure:
      <your_directory>/
      ├── script.py
      └── input/
      ├── theme1.zip (or extracted folder theme1)
      └── theme2.zip (or extracted folder theme2)

    2. Place your themes: Put the two theme ZIP files or extracted folders inside the input directory.

    3. Run the script: Execute the script to generate the comparison report.

    The script will generate a report file named in the format report_<theme1_name>vs<theme2_name>_yyyymmdd_hhmm.txt in the same directory as the script.

    Output

    The output report includes:

    • Missing files: Files present in one theme but missing in the other.
    • Added files: Files added in one theme compared to the other.
    • Modified files: Detailed differences for files that have been modified, using a unified diff format.

    Example Output

    Format of differences:
    @@ -a,b +c,d @@
    a: Starting line number in the original file
    b: Number of lines in the block from the original file
    c: Starting line number in the modified file
    d: Number of lines in the block from the modified file
    -: Lines removed from the original file
    +: Lines added to the modified file

    Missing files in theme2:
    file1.php
    file2.css

    Added files in theme2:
    file3.js
    file4.html

    Modified file: style.css
    — theme1/style.css
    +++ theme2/style.css
    @@ -15,7 +15,7 @@

    • old line 15
    • new line 15
      unchanged line 16
      unchanged line 17
      unchanged line 18
      @@ -25,4 +25,4 @@
    • old line 25
    • new line 25

    Contributing

    Contributions are welcome! Please fork this repository and submit a pull request for any improvements or bug fixes.

    License

    This project is licensed under the MIT License. See the LICENSE file for more details.

    Contact

    For any questions or suggestions, please contact Giovanni Sacheli @ evemilano.com.

    Visit original content creator repository
    https://github.com/evemilano/theme_comparator

  • jenkins-pipeline-immut

    Test Pipeline

    Jenkins Declarative Pipeline Example (with packer, terraform, aws, ansible)

    NOTE: This can be used with my custom Jenkins containers : jenkins docker alpine and jenkins docker debian

    Details

    Contains simple Jenkins pipeline that does the following:

    • Create a custom AWS AMI using Packer; Ansible provisioner to config the image as simple apache static site
    • Terraform creates the infra on AWS (choose either single webserver OR ASG/ELG setup)
    • Options to tear down TF infra and Packer AMI when done to avoid charges and stay in the good blessings of the AWS free tier

    Setup

    To get started do the following (this isn’t step by step so it assumes some very basic Jenkins knowledge):

    1. Create a new declarative pipeline in jenkins
    2. Clone the repo https://github.com/brian-provenzano/jenkins-pipeline-immut locally on your dev env, create your own repo in github, git init, add, commit, push origin to your repo to get setup. Setup your access using keys as needed.
    3. Add Credentials parameter that contains your AWS access and secret keys. Use the “usernamne/password” option.

    credentials alt

    1. Setup the job to pull the repo for the Jenkinsfile (pipeline). Add your keys as needed (from Step 2)

    git Jenkinsfile alt

    1. You should end up with something that looks like this when you view the job (after the initial run/pull).

    job Build parameters alt

    Pipeline Job parameters / options:

    • Cleanup TF infra when done (boolean) *
    • Cleanup custom AMI when done (boolean) *
    • Use jenkins credentials store for AWS credentials
    • Deploy as ASG/ELB or simple web server instance; default is false for simple web instance (boolean)
    • How long to sleep job after TF creates infra so we can test/view what was done. *

    NOTE: *These steps mainly there since this is a lab/test to make sure costs are low and/or stay in free tier.

    TODO

    • packer build AMI in AWS with code baked (build etc); currently only configures a static instance of Apache
    • seperate out the terraform, ansible, packer Iaac/config into seperate repos and pull those directly as part of pipeline instead of packing into one repo. Currently setup this way for easy startup / testing…
    • extend ansible configs to install real systems, maybe use RDS (via TF) etc.
    Visit original content creator repository https://github.com/brian-provenzano/jenkins-pipeline-immut
  • thread-pool-benchmark

    Thread Pool Benchmark

    A C++ Thread Pool Colosseum,
    where various C++ thread pool implementations fight for glory and fortune.

    Install

    git clone --recursive https://github.com/Red-Portal/thread-pool-benchmark.git
    cmake -G "<Generator>"
    make -j4

    Gladiators

    Fight Events

    • 1024 * 1024 Matrix Multiplication
    • 2048 * 2048 Fluid Solver

    Example Results

    Intel Core i7-7700HQ, Manjaro Linux, clang 6.0.0

    matrix multiplication forking joining total
    C++ thread 473.692266ms 14.826739ms 488.519005ms
    Sean Parent Naive 0.312164ms 423.402574ms 423.714738ms
    Sean Parent Multiqueue 0.314355ms 472.653375ms 472.967730ms
    Sean Parent Work Stealing 0.292475ms 443.798525ms 444.091000ms
    progschj’s ThreadPool 0.286827ms 562.374141ms 562.660968ms
    Grand Central Dispatch 0.374894ms 423.493407ms 423.868301ms
    Thread Building Blocks 1.042765ms 488.946873ms 489.989638ms



    fluid solver forking joining total
    C++ thread 1252.285177ms 375.896608ms 1628.181785ms
    Sean Parent Naive 19.664301ms 1149.353465ms 1169.017766ms
    Sean Parent Multiqueue 6.706725ms 1162.610971ms 1169.317696ms
    Sean Parent Work Stealing 9.154444ms 1149.240601ms 1158.395045ms
    progschj’s ThreadPool 7.787490ms 1559.932310ms 1567.719800ms
    Grand Central Dispatch 9.418396ms 1161.239495ms 1170.657891ms
    Thread Building Blocks 10.892387ms 1224.662278ms 1235.554665ms

    Contributions

    All contributions are welcome,
    including new types of benchmark problems and publicly available thread pool implementations

    LICENSE

    thread-pool-benchmark, a C++ Thread Pool Colosseum
    Copyright (C) 2018  Red-Portal
    
    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.
    
    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.
    
    You should have received a copy of the GNU General Public License
    along with this program.  If not, see <http://www.gnu.org/licenses/>.
    

    Visit original content creator repository
    https://github.com/Red-Portal/thread-pool-benchmark

  • thread-pool-benchmark

    Thread Pool Benchmark

    A C++ Thread Pool Colosseum,
    where various C++ thread pool implementations fight for glory and fortune.

    Install

    git clone --recursive https://github.com/Red-Portal/thread-pool-benchmark.git
    cmake -G "<Generator>"
    make -j4

    Gladiators

    Fight Events

    • 1024 * 1024 Matrix Multiplication
    • 2048 * 2048 Fluid Solver

    Example Results

    Intel Core i7-7700HQ, Manjaro Linux, clang 6.0.0

    matrix multiplication forking joining total
    C++ thread 473.692266ms 14.826739ms 488.519005ms
    Sean Parent Naive 0.312164ms 423.402574ms 423.714738ms
    Sean Parent Multiqueue 0.314355ms 472.653375ms 472.967730ms
    Sean Parent Work Stealing 0.292475ms 443.798525ms 444.091000ms
    progschj’s ThreadPool 0.286827ms 562.374141ms 562.660968ms
    Grand Central Dispatch 0.374894ms 423.493407ms 423.868301ms
    Thread Building Blocks 1.042765ms 488.946873ms 489.989638ms



    fluid solver forking joining total
    C++ thread 1252.285177ms 375.896608ms 1628.181785ms
    Sean Parent Naive 19.664301ms 1149.353465ms 1169.017766ms
    Sean Parent Multiqueue 6.706725ms 1162.610971ms 1169.317696ms
    Sean Parent Work Stealing 9.154444ms 1149.240601ms 1158.395045ms
    progschj’s ThreadPool 7.787490ms 1559.932310ms 1567.719800ms
    Grand Central Dispatch 9.418396ms 1161.239495ms 1170.657891ms
    Thread Building Blocks 10.892387ms 1224.662278ms 1235.554665ms

    Contributions

    All contributions are welcome,
    including new types of benchmark problems and publicly available thread pool implementations

    LICENSE

    thread-pool-benchmark, a C++ Thread Pool Colosseum
    Copyright (C) 2018  Red-Portal
    
    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.
    
    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.
    
    You should have received a copy of the GNU General Public License
    along with this program.  If not, see <http://www.gnu.org/licenses/>.
    

    Visit original content creator repository
    https://github.com/Red-Portal/thread-pool-benchmark

  • graph-tools

    graph-tools

    Tools for efficiently detecting subgraph isomorphisms and graph identity using Ullman and NAUTY algorithms in Common Lisp

    (require 'graph-tools)
    (in-package :graph-tools)
    ;;
    ;;
    
    (let* ((graph  (make-graph '((0 1) (0 2) (1 2) (0 3) (2 4))))
            ;; graph:
            ;;           1
            ;;          / \
            ;;         0 - 2
            ;;          \   \
            ;;           3   4
           (subgraph (make-graph '((0 1) (0 2) (1 2))))
            ;; subgraph:
            ;;           1
            ;;          / \
            ;;         0 - 2
           (isomorphisms (find-subgraph-isomorphisms +graph2+ +graph3+)))
      (print isomorphisms)
    )
    ;; 6 isomorphisms:
    #(
      #(0 1 2)
      #(0 2 1)
      #(1 2 0)
      #(1 0 2)
      #(2 0 1)
      #(2 1 0)
    )
    
    (let* ((g (make-graph '((0 1) (0 2) (1 2) (0 3) (2 4))))
      (destructuring-bind (canonical-g reordering) 
        (canonical-graph g)
        (print canonical-g) ;; a graph with identical structure to g
        (print reordering) ;; array of length 5 mapping verticies in canonical-g to g
      )
    ) 

    Graph functions

    (make-graph '((0 1) (0 2) (1 2))) ;; an undirected trianglular graph
    (make-graph '((0 1) (1 2) (2 0)) t) ;; directed triangular graph
    
    (directedp g) ;; is graph directed or undirected?
    
    (graph-directed-edge-p g i1 i1) ;; t if an edge exists from i1 to i2
    (setf (graph-direvted-edge-p g i1 i1) t) ;; connects i1 to i2
    
    (graph-vertex-count g) ;; returns # of verticies
    
    (vertex-paths g v1 v2 &key (max-steps -1) (ignore nil)) 
      ;; returns a list of paths, where each path is a list of verticies starting at v1 and ending in v2
    
    
    (coerce-to-undirected-graph g) ;; adds reverse edges to a directed graph, making it undirected
    
    
    (graph-add-edges g edge-specs)
    (graph-delete-edges g edge-specs) 
    
    (graph-extend-verticies g n) ;; adds n unconnected verticies to the graph
    (graph-delete-verticies g vlist &key not)
    (graph-delete-verticies-if g pred &key not)
    
    (graph-add-graph g1 g2 &optional g1-g2-edges g2-g2-edges) 
      ;; adds two graphs and new edges between the two graphs
      ;; e.g., (graph-add-graph g1 g2 :g1-g2-edges '((0 1)) 
                                      :g2-g1-edges '((1 0)))
      ;;        creates an undirected edge between g1vertex 0 to g2vertex 1
    
    (matrix-of g) ;; gets the underlying bit matrix representing the graph
    
    (reorder-graph g #(2 3 0 1)) ;; returns a new graph with same structure
                                ;; but with verticies reordered
    
    (transpose g) ;; destructively transpose graph g - reverses directed edges
    
    (unconnected-subgraphs g &key (verticies t) output-connected-p)
        ;; Returns a list of subgraphs of g which are disconnected from each other.
        ;; G is a graph
        ;; OUTPUT-CONNECTED is a boolean which if set causes the function to consider
        ;; verticies to be connected when they share an output vertex.  This setting
        ;; only has meaning for directed graphs, since all edges in undirected graphs 
        ;; are bidirectional.  If OUTPUT-CONNECTED-P is NIL, shared output verticies do
        ;; not cause their inputs to be in a connected group; the s
    
    (connected-vertex-groups g &key (verticies t) (max-steps -1) (ignore nil))
      ;; computes the sets of verticies which are connected (by output edges)
    
    

    Ullman subgraph isomorphism functions

    Graphs are represented as bitmatricies (vectors of integers treated as bitfields)

    find-subgraph-isomorphisms

    (find-subgraph-isomorphisms s g &key base-map continue-if vertex-test row-fn)

    Returns a list of where each element represents a subgraph isomorphism.

    Required arguments:

    s – subgraph to find
    g – graph to search

    Optional arguments:

    base-map – allowed mappings from subgraph s verticies to graph g verticies. This parameter can be used to only allow matches between particular verticies. This is a vector where each index represents the corresponding vertex in the subgraph s, and the value is a bitmatrix where each set bit represents an allowed isomorphism to the graph g. An entry containing all bits set means that all mappings are possible; An entry where all bits are 0 means that no mappings are possible between that subgraph vertex and any graph vertex.

    continue-if – lambda which which takes an isomorphism as an argument and returns two booleans (continuep collectp). If collectp is true, the isomorphism is added to the list of returned isomorphisms. If continuep is true, search continues for more isomorphisms.

    vertex-test – predicate used to limit which vertexes in s can match to vertexes in g. It takes arguments (s svertex g gvertex), where s and g are the subgraph and graph and verticies being tests, and returns NIL if svertex cannot map to gvertex.

    row-fn – is an alternative way of computing which mappings are possible, and takes arguments (s svertex g) and returns an integer bitfield representing the indicies of g to which svertex may be mapped.

    Note: if neither vertex-test nor row-fn are provided, a default vertex-test is used which only allows sverticies to map to gverticies with an equal or greater number of outgoing edges.

    find-subgraph-isomorphism-maps

    (find-subgraph-isomorphism-maps s g &key base-map (continue-if t) vertex-test row-fn)

    Identical to find-subgraph-isomorphisms, but returns a list of bit-vectors instead of a list of integer vectors. Useful if you want to avoid the additional overhead of translating from bit-vectors to integer arrays.

    Visit original content creator repository
    https://github.com/aneilbaboo/graph-tools

  • ctc-commands

    ctc-commands

    Setup

    This repo contains model trained on version 1 of audio commands dataset [1]. The dataset contains total of 30 words pronounced by different English speakers and was divided into train/val/test sets in accordance with [1], exact filenames can be found in data_division for the ease of use. Raw audio files were preprocessed by computing their spectgorams which were than fed to BLSTM network. The network itself was trained to output the distributions of vocabulary characters(24 in total) for each spectrogram time step, which were than optimized to minimize the CTC loss [2, 3]. Inference was done by greedy, beam and constrained [3] search algorithms.

    Results

    Trained model was able to achieve 6.93%/8.92%/9.32% CER(character error rate) on train/val/test respectively using beam search with width of 5. The greedy search and constrained search gave 9.64% CER and 8.31% CER on test set respectively. Constrained search was pretty helpful when correcting small grammar mistakes, but as it turns out the majority of CER was caused by whole words confounded with other(e.g. “bird” <-> “bed”). In order to check how model behaves without relying on any grammar infomation further experiments were carried out using greedy search. It was preferred over beam search as latter didn’t give any substantial improvement in CER and was several times slower. When it comes to WER(word error rate) the model was able to achieve 10.61% WER on test set using constraint search.

    Below plot shows CER on test set by words. Ones of the hardest words were the short ones such as “off”, “up”, “go”, “no”, but also similarly pronounced ones as e.g. “bird” – “bed”, “dog” – “down” or “tree” – “three”.

    drawing

    The next plot demonstrates character probability distributions in time for recording of word “happy”.

    drawing

    In general the trained model(can be found in model.zip) coped with its task pretty well even though it overfitted to train set a bit. The task itself is hard even for humans, as the context is short and missing even single character can lead to word misunderstanding. It is also worth mentioning that experiments with unidirectional LSTM were conducted, but this kind of recurrent neural network was not able to deliver any meaningful results givin CER above 100%.

    References:

    [1] Warden P., Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition, https://arxiv.org/pdf/1804.03209.pdf

    [2] Graves A., Fernandez S., Gomez F. and Schmidhuber J., Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks, https://www.cs.toronto.edu/~graves/icml_2006.pdf

    [3] Graves A., Supervised Sequence Labelling with Recurrent Neural Networks, https://www.cs.toronto.edu/~graves/preprint.pdf

    Visit original content creator repository https://github.com/gekas145/ctc-commands