ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Transparent compression on folders in EXT4/XFS etc?

    IT Discussion
    compression ext4 xfs zfs btrfs
    5
    23
    19.8k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • dafyreD
      dafyre @1337
      last edited by dafyre

      @Pete-S said in Transparent compression on folders in EXT4/XFS etc?:

      Too bad that wasn't possible.

      BTW looking around I found that Facebook announced last year that they are running btrfs in production on millions of servers.
      https://code.fb.com/open-source/linux/

      I wonder how often they have to restore a system from backup because BtrFS decided to crap out...

      Edit: Maybe they can fix that.

      1 Reply Last reply Reply Quote 1
      • dafyreD
        dafyre
        last edited by

        I'm actually toying around with ZFS at the house now using it's lz4 compression.... Getting ~1.8x compression...

        Doesn't seem to affect the speed of anything either.

        root@bigcomputer ~ # zfs get compress
NAME            PROPERTY     VALUE     SOURCE
TestVM          compression  lz4       local
TestVM/Storage  compression  lz4       inherited from TestVM

root@bigcomputer ~ # zfs get compressratio
NAME            PROPERTY       VALUE  SOURCE
TestVM          compressratio  1.87x  -
TestVM/Storage  compressratio  1.87x  -
root@bigcomputer ~ #
        scottalanmillerS 1 Reply Last reply Reply Quote 0
        • scottalanmillerS
          scottalanmiller @dafyre
          last edited by

          @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

          Doesn't seem to affect the speed of anything either.

          SSD or Winchester?

          dafyreD 1 Reply Last reply Reply Quote 0
          • dafyreD
            dafyre @scottalanmiller
            last edited by dafyre

            @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

            @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

            Doesn't seem to affect the speed of anything either.

            SSD or Winchester?

            Winchester.

            Edit: I'm going to put my Zabbix instance on it later and see how it does.

            scottalanmillerS 2 Replies Last reply Reply Quote 0
            • scottalanmillerS
              scottalanmiller @dafyre
              last edited by

              @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

              @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

              @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

              Doesn't seem to affect the speed of anything either.

              SSD or Winchester?

              Winchester.

              Oh, would expect it to speed up.

              dafyreD 1 Reply Last reply Reply Quote 0
              • scottalanmillerS
                scottalanmiller @dafyre
                last edited by

                @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                Edit: I'm going to put my Zabbix instance on it later and see how it does.

                Databases should not be compressed!

                dafyreD 2 Replies Last reply Reply Quote 0
                • dafyreD
                  dafyre @scottalanmiller
                  last edited by

                  @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

                  @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                  Edit: I'm going to put my Zabbix instance on it later and see how it does.

                  Databases should not be compressed!

                  In an ideal world. 🙂

                  The Database itself is not compressed... the entire VM is. 🙂

                  9GB down to 3.2GB. The system runs pretty good. I can't say it's noticeably faster, but it's definitely not slower.

                  scottalanmillerS 2 Replies Last reply Reply Quote 0
                  • scottalanmillerS
                    scottalanmiller @dafyre
                    last edited by

                    @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                    The Database itself is not compressed... the entire VM is.

                    Same issues 🙂 It doesn't care where the compression sits.

                    1 Reply Last reply Reply Quote 0
                    • scottalanmillerS
                      scottalanmiller @dafyre
                      last edited by

                      @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                      9GB down to 3.2GB. The system runs pretty good. I can't say it's noticeably faster, but it's definitely not slower.

                      That's cool, a lot of savings for sure.

                      1 Reply Last reply Reply Quote 0
                      • dafyreD
                        dafyre @scottalanmiller
                        last edited by

                        @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

                        @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                        Edit: I'm going to put my Zabbix instance on it later and see how it does.

                        Databases should not be compressed!

                        Details as to why databases should not be compressed?

                        scottalanmillerS 1 Reply Last reply Reply Quote 0
                        • dafyreD
                          dafyre @scottalanmiller
                          last edited by

                          @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

                          @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                          @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

                          @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                          Doesn't seem to affect the speed of anything either.

                          SSD or Winchester?

                          Winchester.

                          Oh, would expect it to speed up.

                          So far the only side effects that I'm noticing are the increased amounts of free space and a a (significant?) increase in the number or processes on the system... not necessarily system load, either... I wouldn't have noticed it if a Zabbix trigger hadn't hollered about it.

                          1 Reply Last reply Reply Quote 0
                          • scottalanmillerS
                            scottalanmiller @dafyre
                            last edited by

                            @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                            @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

                            @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                            Edit: I'm going to put my Zabbix instance on it later and see how it does.

                            Databases should not be compressed!

                            Details as to why databases should not be compressed?

                            Basically because they are always open and written to incrementally. They aren't loaded and rewriteen like most files are. And they tend to be very large, so a very intensive usage pattern.

                            dafyreD 1 Reply Last reply Reply Quote 0
                            • dafyreD
                              dafyre @scottalanmiller
                              last edited by

                              @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

                              @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                              @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

                              @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                              Edit: I'm going to put my Zabbix instance on it later and see how it does.

                              Databases should not be compressed!

                              Details as to why databases should not be compressed?

                              Basically because they are always open and written to incrementally. They aren't loaded and rewriteen like most files are. And they tend to be very large, so a very intensive usage pattern.

                              True. But this compression is being done on the Host OS, not inside the Zabbix VM. I wonder what kind of strangeness this can cause. I don't have a lot of traffic on this particular server.

                              scottalanmillerS 1 Reply Last reply Reply Quote 0
                              • scottalanmillerS
                                scottalanmiller @dafyre
                                last edited by

                                @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                                @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

                                @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                                @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

                                @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                                Edit: I'm going to put my Zabbix instance on it later and see how it does.

                                Databases should not be compressed!

                                Details as to why databases should not be compressed?

                                Basically because they are always open and written to incrementally. They aren't loaded and rewriteen like most files are. And they tend to be very large, so a very intensive usage pattern.

                                True. But this compression is being done on the Host OS, not inside the Zabbix VM. I wonder what kind of strangeness this can cause. I don't have a lot of traffic on this particular server.

                                That doesn't affect anything. Compression is compression.

                                dafyreD 1 Reply Last reply Reply Quote 0
                                • dafyreD
                                  dafyre @scottalanmiller
                                  last edited by

                                  @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

                                  @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                                  @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

                                  @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                                  @scottalanmiller said in Transparent compression on folders in EXT4/XFS etc?:

                                  @dafyre said in Transparent compression on folders in EXT4/XFS etc?:

                                  Edit: I'm going to put my Zabbix instance on it later and see how it does.

                                  Databases should not be compressed!

                                  Details as to why databases should not be compressed?

                                  Basically because they are always open and written to incrementally. They aren't loaded and rewriteen like most files are. And they tend to be very large, so a very intensive usage pattern.

                                  True. But this compression is being done on the Host OS, not inside the Zabbix VM. I wonder what kind of strangeness this can cause. I don't have a lot of traffic on this particular server.

                                  That doesn't affect anything. Compression is compression.

                                  I'll find out what kind of performance hits I take with it on ZFS. So far, I'm seeing some nice space savings and no problems with anything else.

                                  1 Reply Last reply Reply Quote 0
                                  • 1
                                  • 2
                                  • 2 / 2
                                  • First post
                                    Last post